Events

CI/CD for Data - Building Data Development Environment with lakeFS - devopsdays Buffalo 2023

devopsdays.org

A property of data pipelines one might observe is that they rarely stay still. Instead, there are near-constant updates to some aspect of the infrastructure they run on, or in the logic they use to transform data, below are two examples. To efficiently apply the necessary changes to a pipeline requires running it parallel to production to test the effect of a change. Most data engineers would agree that the best way…

Visit Site