Release Notes – 0.29.0

We ship fast and often at Sematic. We just cut 0.29.0 with a number of new features, reliability improvements, and bug fixes. Let's dive in!

Pipeline metrics

We added three top-level metrics to all pipelines: run count, success rate, and average runtime. Users can easily get a sense of the more robust or fragile pipelines and how fast they run.

Function timeouts

Some Functions need costly resources (e.g. GPUs), and they might be querying external services (e.g. a data warehouse, a third-party API). When those take too long, the dollars keep rising. To give users more control, we introduce Function timeouts.

Pass an integer value in minutes, and if the Function takes longer, it will be killed. The Function's Run will be marked as failed, and the rest of pipeline will be canceled. This works for both Inline and Standalone Functions, and for local and cloud execution.

Kubernetes Pod lifecycle

To make debugging easier and faster, Sematic now has a new Pod tab that shows the lifecycle of the Pod that was spun up specifically for the selected run.

Renaming Inline to Standalone

We heard your feedback about the confusion regarding Inline Functions. When using cloud execution, all Functions are Inline by default, which means that they run in the same Kubernetes Pod as the overall pipeline resolver. When a Function needs its own resources (e.g. high memory, or GPUs), they used to be called Non-Inline. We simplified this by calling those Standalone Functions, since they run in a dedicated/standalone Kubernetes Pod. We also renamed the corresponding decorator argument from inline to standalone.

The default behavior has not changed, which is that Functions run inline (standalone=False).

As always, our deprecation window spans two releases. Therefore, the inline decorator argument will be supported until 0.31.0.

Customize local storage location

When running the Sematic server on you local machine with $ sematic start, artifact data gets stored by default to ~/.sematic/data. When deploying the Sematic server on a remove machine (either as a standalone Docker container or on Kubernetes), typically users select a cloud storage solution to persist artifacts (e.g. AWS S3). We learned recently that this is not always the case. Some users may want to deploy Sematic on-premise, in which case using a cloud storage solution makes less sense.

To accommodate this, we made the local storage location customizable. If you are deploying Sematic as a standalone Docker container, simply set the path with:

$ sematic settings set -p sematic.plugins.storage.local_storage.LocalStorage \
    LOCAL_STORAGE_PATH /path/to/location

If you are deploying Sematic on Kubernetes with our Helm chart, simply add this section to your values.yaml file:

local_storage:
  enabled: true
  local_storage_path: /path/to/storage

Backward log scrolling

Function logs can be extra long, and therefore need to be paginated. We already had a way for users to jump straight to the end of logs, now we are adding the ability to scroll logs backwards, and jump back to the top.

Enable Ray autoscaling

After shipping our Ray integration, we are now enabling autoscaling to give you more control over your Ray clusters. By passing a min_workers and a max_workers to your ScalingGroup, users can leverage autoscaling Ray cluster.

Garbage collection cron job

Kubernetes is a tough beast. Sometimes certain Pods are left hanging, or fail with obscure reasons. When this happens, metadata stored in the Sematic database can be left in a weird non-terminal state. To remedy this, we are adding a cron job to periodically clean up these non-terminal states and make sure everything looks good.

This change ships automatically with out Helm chart so no change needed on your end.

Upgrade

To upgrade, simply issue:

and to upgrade your Kubernetes deployment:

As always, if you hit any issues or have any questions, ask us on our Discord server.