Skip to content

HPC

It is all about I/O!

Read the full blog post about GPU-native data loading here: https://xarray.dev/blog/gpu-pipeline

Data loading is a major bottleneck in AI/ML workflows, especially when working with large geoscientific datasets. At a hackathon, we explored how GPU-native data loaders using Zarr v3, CuPy, and DALI can help overcome this bottleneck and accelerate end-to-end workflows.