Release Notes – 0.27.0

We have spent the better part of Q1 '23 onboarding users and listening to the requests and feedback. This is a crucial part of our product development process as it helps us prioritize our roadmap, and ensure Sematic is the most useful it can be.

This week, we released version 0.27.0, with many new features and improvements. Here's a rundown of the most exciting ones.

Python 3.10 support

Python 3.10 was released over a year ago and is quickly gaining ground amongst our user base. With this release, users can use Sematic in their 3.10 codebase.

Ray integration

Ray is a very popular open-source framework to distribute workloads horizontally. It enables large-scale parallelization of resource-hungry workloads such as data processing, model training, etc.

With Sematic’s Ray integration, you can easily spin up and down Ray clusters on-demand and scale your workloads.

See our example pipelines:

• CIFAR classifier with distributed training: Sematic example
• PyTorch Lightning distributed ResNet: Sematic example

Run caching

When iterating on a pipeline step, upstream steps typically do not change. For example, you may be iterating on your model evaluation strategy, and you don’t want to prepare the data and retrain the model again every time you run the entire pipeline.

Sematic lets you mark a particular function as cached so that if its inputs do not change, a cached output will be used. This is huge productivity booster, and cost cutter. Read the docs here.

Image type

Sematic strives to make it easy to visualize in the dashboard any type of object returned by your functions.

We just added the Image type so that you can return image files by themselves, or as part of a greater output object (e.g. a dataclass, a list, a tuple, a dictionary, etc.).

It’s as easy as using Sematic’s Image type. See full docs here.

The above function output will render in the Dashboard as shown below.

Run search

To make it easier to find exactly the information you are looking for, we added a Run Search functionality.

You can search by name, import path, tags, etc.

We’ll add more functionalities over time to filter by run status, user, etc.

And many more...

Here is a sample of other features we added recently:

• Faster Bazel builds
  Sematic uses Bazel to package dependencies at runtime and ship them to your Kubernetes cluster. We added ways to layer your Docker images so that it is faster to build them when you submit pipelines to run in the cloud.
  ‍
• Simplifying run URLs
  In order to make it possible to construct run URLs programmatically, we simplified our URL pattern to be simply https://<hostname>/runs/<runID>
  ‍
• Teeing container logs
  When you run a pipeline in the cloud, container logs are no longer only captured for display in the Sematic dashboard. They are also piped to the container’s stdout so that tools like Loki and kubectl can access them as well.
  ‍
• Scaling the server
  We made it possible to scale the Sematic API server horizontally by breaking out the Web Socket server into into own singleton server.

Get started

Get started with Sematic today!

$ pip install sematic
$ sematic start
$ sematic run examples/mnist/pytorch

Read our documentation and join us on Discord.