đ Blogs & Articles â Awesome Time Series
Loading postsâŠ
Loading postsâŠ
51 posts, articles, and resources from across the field.
Innovative AI techniques are being developed to enhance weather prediction and disaster management.
3 postsResearch is focusing on improving temporal awareness in various AI models for better understanding and prediction.
3 postsWhy it matters â TopoPrimer introduces a novel approach by incorporating the topological structure of time series data into forecasting models, which can enhance predictive accuracy and stability, particularly in seasonal contexts.
Why it matters â This post highlights the importance of incorporating humidity data into meteorological models, which is crucial for improving the accuracy of weather forecasts. Understanding the role of water vapor can lead to better predictive models for storm behavior.
Why it matters â The application of AI in flash flood forecasting demonstrates the potential of machine learning techniques to enhance real-time prediction capabilities, which is vital for urban disaster management. This research can inform the development of more resilient city infrastructures.
Why it matters â Mamba presents a novel approach to sequence modeling through State Space Models, which could offer a more efficient alternative to Transformers, particularly for long time series data. This has implications for various applications in time series analysis and forecasting.
Why it matters â Aurora 1.5's enhancements in temporal resolution and probabilistic forecasting make it a significant advancement for climate and weather modeling, allowing for more accurate predictions in real-world applications. This can improve decision-making in sectors reliant on weather data.
Why it matters â The PEVA model's ability to predict future video frames based on past actions introduces a new dimension to time series forecasting in visual data, which can enhance applications in video analysis and surveillance. This technique could be pivotal for developing more advanced predictive models in dynamic environments.
Why it matters â This post addresses the critical need for temporal awareness in multimodal models, which is essential for accurately interpreting sequences of events in time series data. Enhancing temporal reasoning can lead to better performance in tasks that rely on chronological order.
Why it matters â The discussion on the decreasing costs of AI technologies highlights the accessibility of advanced data systems, which can empower researchers to innovate in time series analysis without prohibitive financial barriers. This democratization of AI tools can lead to broader applications in various fields.
Why it matters â This post discusses the efficiency of diffusion language models in comparison to autoregressive models, providing insights that could influence the design of time series forecasting models.
Why it matters â Residual context in diffusion models suggests a novel way to improve parallel processing in time series data, potentially leading to faster and more accurate predictions.
Why it matters â Celebrating the achievements of Ph.D. graduates in AI research underscores the importance of academic contributions to the field, particularly in developing new methodologies for time series analysis. Their work can inspire future innovations in this area.
Why it matters â Adaptive parallel reasoning introduces a new paradigm for efficiently scaling inference tasks, which can significantly enhance the processing of large time series datasets. This technique can optimize resource allocation, improving computational efficiency in time series analysis.
Why it matters â Understanding interactions within large language models is crucial for improving their interpretability, which can inform better modeling techniques for time series data. This research can lead to more transparent and reliable AI systems in various applications.
Why it matters â The information-driven design of imaging systems emphasizes the importance of noise modeling in time series data collection, which can enhance the accuracy of measurements in various applications. This approach can lead to better data quality in time series analysis.
Why it matters â Introducing a reinforcement learning algorithm that does not rely on temporal difference learning can provide new insights into scalable methods for time series prediction. This alternative approach could lead to more robust models in dynamic environments.
Why it matters â The exploration of hybrid preferences in routing instances for human vs. AI feedback highlights the evolving landscape of model training, which can influence the effectiveness of time series models. Understanding these dynamics can enhance the integration of human insights into AI systems.
Why it matters â Investigating pretraining dynamics and stability in language models can inform best practices for developing time series models, particularly in terms of robustness and reliability. This research contributes to the foundation of open-source efforts in model training.
NVIDIA XR AI is now available in public beta, giving developers a framework for building multimodal AI agents for AR glasses and XR devices.  
Apple is presenting new research at the annual IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), which takes place in person in Denver at the Colorado Convention Center from June 3 to June 7. We are proud to sponsor the conference, which brings together the scientific and industrial research communities in computer vision and pattern recognition. Below is an overview of Appleâs participation at CVPR 2026.
Streaming vision-language models (VLMs) continuously generate responses given an instruction prompt and an online stream of input frames. This is a core mechanism for real-time visual assistants. Existing VLM frameworks predominantly assess models in offline settings. In contrast, the performance of a streaming VLM depends on additional metrics beyond pure video understanding, including proactiveness, which reflects the timeliness of the modelâs responses, and consistency, which captures the robustness of its responses over time. To address this limitation, we propose VSAS-Bench, a newâŠ
Image captioning is one of the most fundamental tasks in computer vision. Owing to its open-ended nature, it has received significant attention in the era of multimodal large language models (MLLMs). In pursuit of ever more detailed and accurate captions, recent work has increasingly turned to reinforcement learning (RL). However, existing captioning-RL methods and evaluation metrics often emphasize a narrow notion of caption quality, inducing trade-offs across core dimensions of captioning. For example, utility-oriented objectives can encourage noisy, hallucinated, or overlong captions thatâŠ
Normalizing Flows (NFs) are a classical family of likelihood-based methods that have received revived attention. Recent efforts such as TARFlow have shown that NFs are capable of achieving promising performance on image modeling tasks, making them viable alternatives to other methods such as diffusion models. In this work, we further advance the state of Normalizing Flow generative models by introducing iterative TARFlow (iTARFlow). Unlike diffusion models, iTARFlow maintains a fully end-to-end, likelihood-based objective during training. During sampling, it performs autoregressive generationâŠ
True spatial intelligence for multimodal agents transcends low-level geometric perception, evolving from knowing where things are to understanding what they are for. While existing benchmarks, such as VSI-Bench, effectively evaluate this foundational geometric stage, they fall short of probing the higher-order cognitive abilities essential for grounded intelligence. To bridge this gap, we introduce the Spatial-Functional Intelligence Benchmark (SFI-Bench), a video-based benchmark with over 1700 questions derived from diverse, egocentric indoor video scans. SFI-Bench is designed toâŠ
AI agent systems today juggle separate models for vision, speech and language â losing time and context as they pass data from one model to the other. Unveiled today, NVIDIA Nemotron 3 Nano Omni is an open multimodal model that brings these capabilities together into one system, enabling agents to deliver faster, smarter responses with […]
GRASP is a new gradient-based planner for learned dynamics (a âworld modelâ) that makes long-horizon planning practical by (1) lifting the trajectory into virtual states so optimization is parallel across time, (2) adding stochasticity directly to the state iterates for exploration, and (3) reshaping gradients so actions get clean signals while we avoid brittle âstate-inputâ gradients through high-dimensional vision models. Large, learned world models are becoming increasingly capable. They can predict long sequences of future observations in high-dimensional visual spaces and generalize across tasks in ways that were difficult to imagine a few years ago. As these models scale, they start to look less like task-specific predictors and more like general-purpose simulators. But having a powerful predictive model is not the same as being able to use it effectively for control/learning/planning. In practice, long-horizon planning with modern world models remains fragile: optimization becomes ill-conditioned, non-greedy structure creates bad local minima, and high-dimensional latent spaces introduce subtle failure modes. In this blog post, I describe the problems that motivated this project and our approach to address them: why planning with modern world models can be surprisingly fragile, why long horizons are the real stress test, and what we changed to make gradient-based planning much more robust. This blog post discusses work done with Mike Rabbat, Aditi Krishnapriyan, Yann LeCun, and Amir Bar (* denotes equal advisorship), where we propose GRASP. What is a world model? These days, the term âworld modelâ is quite overloaded, and depending on the context can either mean an explicit dynamics model or some implicit, reliable internal state that a generative model relies on (e.g. when an LLM generates chess moves, whether there is some internal representation of the board). We give our loose working definition below. Suppose you take actions $a_t \in \mathcal{A}$ and observe states $s_t \in \mathcal{S}$ (images, latent vectors, proprioception). A world model is a learned model that, given the current state and a sequence of future actions, predicts what will happen next. Formally, it defines a predictive distribution on a sequence of observed states $s_{t-h:t}$ and current action $a_t$: \[P_\theta(s_{t+1} \mid s_{t-h:t},\; a_t)\] that approximates the environmentâs true conditional $P(s_{t+1} \mid s_{t-h:t},\; a_t)$. For this blog post, weâll assume a Markovian model $P(s_{t+1} \mid s_{t-h:t},\; a_t)$ for simplicity (all results here can be extended to the more general case), and when the model is deterministic it reduces to a map over states: \[s_{t+1} = F_\theta(s_t, a_t).\] In practice the state $s_t$ is often a learned latent representation (e.g., encoded from pixels), so the model operates in a (theoretically) compact, differentiable space. The key point is that a world model gives you a differentiable simulator; you can roll it forward under hypothetical action sequences and backpropagate through the predictions. Planning: choosing actions by optimizing through the model Given a start $s_0$ and a goal $g$, the simplest planner chooses an action sequence $\mathbf{a}=(a_0,\dots,a_{T-1})$ by rolling out the model and minimizing terminal error: \[\min_{\mathbf{a}} \; \| s_T(\mathbf{a}) - g \|_2^2, \quad \text{where } s_T(\mathbf{a}) = \mathcal{F}_{\theta}^{T}(s_0,\mathbf{a}).\] Here we use $\mathcal{F}^T$ as shorthand for the full rollout through the world model (dependence on model parameters $\theta$ is implicit): \[\mathcal{F}_{\theta}^{T}(s_0, \mathbf{a}) = F_\theta(F_\theta(\cdots F_\theta(s_0, a_0), \cdots, a_{T-2}), a_{T-1}).\] In short horizons and low-dimensional systems, this can work reasonably well. But as horizons grow and models become larger and more expressive, its weaknesses become amplified. So why doesnât this just work at scale? Why long-horizon planning is hard (even when everything is differentiable) There are two separate pain points for the more general world model, plus a third that is specific to learned, deep learning-based models. 1) Long-horizon rollouts create deep, ill-conditioned computation graphs Those familiar with backprop through time (BPTT) may notice that weâre differentiating through a model applied to itself repeatedly, which will lead to the exploding/vanishing gradients problem. Namely, if we take derivatives (note weâre differentiating vector-valued functions, resulting in Jacobians that we denote with $D_x (\cdots)$) with respect to earlier actions (e.g. $a_0$): \[D_{a_0} \mathcal{F}_{\theta}^{T}(s_0, \mathbf{a}) = \Bigl(\prod_{t=1}^T D_s F_\theta(s_t, a_t)\Bigr) D_{a_0}F_\theta(s_0, a_0).\] We see that the Jacobianâs conditioning scales exponentially with time $T$: \[\sigma_{\text{max/min}}(D_{a_0}\mathcal{F}_{\theta}^{T}) \sim \sigma_{\text{max/min}}(D_s F_\theta)^{T-1},\] leading to exploding or vanishing gradients. 2) The landscape is non-greedy and full of traps At short horizons, the greedy solution, where we move straight toward the goal at every step, is often good enough. If you only need to plan a few steps ahead, the optimal trajectory usually doesnât deviate much from âhead toward $g$â at each step. As horizons grow, two things happen. First, longer tasks are more likely to require non-greedy behavior: going around a wall, repositioning before pushing, backing up to take a better path. And as horizons grow, more of these non-greedy steps are typically needed. Second, the optimization space itself scales with horizon: $\mathrm{dim}(\mathcal{A} \times \cdots \times \mathcal{A}) = T\mathrm{dim}(\mathcal{A})$, further expanding the space of local minima for the optimization problem. Distance to goal along the optimal path is non-monotonic, and the resulting loss landscape can be rough. A long-horizon fix: lifting the dynamics constraint Suppose we treat the dynamics constraint $s_{t+1} = F_{\theta}(s_t, a_t)$ as a soft constraint, and we instead optimize the following penalty function over both actions $(a_0,\ldots,a_{T-1})$ and states $(s_0,\ldots,s_T)$: \[\min_{\mathbf{s},\mathbf{a}} \mathcal{L}(\mathbf{s}, \mathbf{a}) = \sum_{t=0}^{T-1} \big\|F_\theta(s_t,a_t) - s_{t+1}\big\|_2^2, \quad \text{with } s_0 \text{ fixed and } s_T=g.\] This is also sometimes called collocation in planning/robotics literature. Note the lifted formulation shares the same global minimizers as the original rollout objective (both are zero exactly when the trajectory is dynamically feasible). But the optimization landscapes are very different, and we get two immediate benefits: Each world model evaluation $F_{\theta}(s_t,a_t)$ depends only on local variables, so all $T$ terms can be computed in parallel across time, resulting in a huge speed-up for longer horizons, and You no longer backpropagate through a single deep $T$-step composition to get a learning signal, since the previous product of Jacobians now splits into a sum, e.g.: \[D_{a_0} \mathcal{L} = 2(F_\theta(s_0, a_0) - s_1).\] Being able to optimize states directly also helps with exploration, as we can temporarily navigate through unphysical domains to find the optimal plan: Collocation-based planning allows us to directly perturb states and explore midpoints more effectively. However, lunch is never free. And indeed, especially for deep learning-based world models, there is a critical issue that makes the above optimization quite difficult in practice. An issue for deep learning-based world models: sensitivity of state-input gradients The tl;dr of this section is: directly optimizing states through a deep learning-based $F_{\theta}$ is incredibly brittle, Ă la adversarial robustness. Even if you train your world model in a lower-dimensional state space, the training process for the world model makes unseen state landscapes very sharp, whether it be an unseen state itself or simply a normal/orthogonal direction to the data manifold. Adversarial robustness and the âdimpled manifoldâ model Adversarial robustness originally looked at classification models $f_\theta : \mathbb{R}^{w\times h \times c} \to \mathbb{R}^K$, and showed that by following the gradient of a particular logit $\nabla f_\theta^k$ from a base image $x$ (not of class $k$), you did not have to move far along $xâ = x + \epsilon\nabla f_\theta^k$ to make $f_\theta$ classify $xâ$ as $k$ (Szegedy et al., 2014; Goodfellow et al., 2015): Depiction of the classic example from (Goodfellow et al., 2015). Later work has painted a geometric picture for whatâs going on: for data near a low-dimensional manifold $\mathcal{M}$, the training process controls behavior in tangential directions, but does not regularize behavior in orthogonal directions, thus leading to sensitive behavior (Stutz et al., 2019). Another way stated: $f_\theta$ has a reasonable Lipschitz constant when considering only tangential directions to the data manifold $\mathcal{M}$, but can have very high Lipschitz constants in normal directions. In fact, it often benefits the model to be sharper in these normal directions, so it can fit more complicated functions more precisely. As a result, such adversarial examples are incredibly common even for a single given model. Further, this is not just a computer vision phenomenon; adversarial examples also appear in LLMs (Wallace et al., 2019) and in RL (Gleave et al., 2019). While there are methods to train for more adversarially robust models, there is a known trade-off between model performance and adversarial robustness (Tsipras et al., 2019): especially in the presence of many weakly-correlated variables, the model must be sharper to achieve higher performance. Indeed, most modern training algorithms, whether in computer vision or LLMs, do not train adversarial robustness out. Thus, at least until deep learning sees a major regime change, this is a problem weâre stuck with. Why is adversarial robustness an issue for world model planning? Consider a single component of the dynamics loss weâre optimizing in the lifted state approach: \[\min_{s_t, a_t, s_{t+1}} \|F_\theta(s_t, a_t) - s_{t+1}\|_2^2\] Letâs further focus on just the base state: \[\min_{s_t} \|F_\theta(s_t, a_t) - s_{t+1}\|_2^2.\] Since world models are typically trained on state/action trajectories $(s_1, a_1, s_2, a_2, \ldots)$, the state-data manifold for $F_{\theta}$ has dimensionality bounded by the action space: \[\mathrm{dim}(\mathcal{M}_s) \le \mathrm{dim}(\mathcal{A}) + 1 + \mathrm{dim}(\mathcal{R}),\] where $\mathcal{R}$ is some optional space of augmentations (e.g. translations/rotations). Thus, we can typically expect $\mathrm{dim}(\mathcal{M}_s)$ to be much lower than $\mathrm{dim}(\mathcal{S})$, and thus: it is very easy to find adversarial examples that hack any state to any other desired state. As a result, the dynamics optimization \[\sum_{t=0}^{T-1} \big\|F_\theta(s_t,a_t) - s_{t+1}\big\|_2^2\] feels incredibly âsticky,â as the base points $s_t$ can easily trick $F_{\theta}$ into thinking itâs already made its local goal.1 1. This adversarial robustness issue, while particularly bad for lifted-state approaches, is not unique to them. Even for serial optimization methods that optimize through the full rollout map $\mathcal{F}^T$, it is possible to get into unseen states, where it is very easy to have a normal component fed into the sensitive normal components of $D_s F_{\theta}$. The action Jacobianâs chain rule expansion is \[\Bigl(\prod_{t=1}^T D_s F_\theta(s_t, a_t)\Bigr) D_{a_0}F_\theta(s_0, a_0).\] See what happens if any stage of the product has any component normal to the data manifold. â© Our fix This is where our new planner GRASP comes in. The main observation: while $D_s F_{\theta}$ is untrustworthy and adversarial, the action space is usually low-dimensional and exhaustively trained, so $D_a F_{\theta}$ is actually reasonable to optimize through and doesnât suffer from the adversarial robustness issue! The action input is usually lower-dimensional and densely trained (the model has seen every action direction), so action gradients are much better behaved. At its core, GRASP builds a first-order lifted state / collocation-based planner that is only dependent on action Jacobians through the world model. We thus exploit the differentiability of learned world models $F_{\theta}$, while not falling victim to the inherent sensitivity of the state Jacobians $D_s F_{\theta}$. GRASP: Gradient RelAxed Stochastic Planner As noted before, we start with the collocation planning objective, where we lift the states and relax dynamics into a penalty: \[\min_{\mathbf{s},\mathbf{a}} \mathcal{L}(\mathbf{s}, \mathbf{a}) = \sum_{t=0}^{T-1} \big\|F_\theta(s_t,a_t) - s_{t+1}\big\|_2^2, \quad \text{with } s_0 \text{ fixed and } s_T=g.\] We then make two key additions. Ingredient 1: Exploration by noising the state iterates Even with a smoother objective, planning is nonconvex. We introduce exploration by injecting Gaussian noise into the virtual state updates during optimization. A simple version: \[s_t \leftarrow s_t - \eta_s \nabla_{s_t}\mathcal{L} + \sigma_{\text{state}} \xi, \qquad \xi\sim\mathcal{N}(0,I).\] Actions are still updated by non-stochastic descent: \[a_t \leftarrow a_t - \eta_a \nabla_{a_t}\mathcal{L}.\] The state noise helps you âhopâ between basins in the lifted space, while the actions remain guided by gradients. We found that specifically noising states here (as opposed to actions) finds a good balance of exploration and the ability to find sharper minima.2 2. Because we only noise the states (and not the actions), the corresponding dynamics are not truly Langevin dynamics. â© Ingredient 2: Reshape gradients: stop brittle state-input gradients, keep action gradients As discussed, the fragile pathway is the gradient that flows into the state input of the world model, \(D_s F_{\theta}\). The most straightforward way to do this initially is to just stop state gradients into \(F_{\theta}\) directly: Let $\bar{s}_t$ be the same value as $s_t$, but with gradients stopped. Define the stop-gradient dynamics loss: \[\mathcal{L}_{\text{dyn}}^{\text{sg}}(\mathbf{s},\mathbf{a}) = \sum_{t=0}^{T-1} \big\|F_\theta(\bar{s}_t, a_t) - s_{t+1}\big\|_2^2.\] This alone does not work. Notice now states only follow the previous stateâs step, without anything forcing the base states to chase the next ones. As a result, there are trivial minima for just stopping at the origin, then only for the final action trying to get to the goal in one step. Dense goal shaping We can view the above issue as the goalâs signal being cut off entirely from previous states. One way to fix this is to simply add a dense goal term throughout prediction: \[\mathcal{L}_{\text{goal}}^{\text{sg}}(\mathbf{s},\mathbf{a}) = \sum_{t=0}^{T-1} \big\|F_\theta(\bar{s}_t, a_t) - g\big\|_2^2.\] In normal settings this would over-bias towards the greedy solution of straight chasing the goal, but this is balanced in our setting by the stop-gradient dynamics lossâs bias towards feasible dynamics. The final objective is then as follows: \[\mathcal{L}(\mathbf{s},\mathbf{a}) = \mathcal{L}_{\text{dyn}}^{\text{sg}}(\mathbf{s},\mathbf{a}) + \gamma \, \mathcal{L}_{\text{goal}}^{\text{sg}}(\mathbf{s},\mathbf{a}).\] The result is a planning optimization objective that does not have dependence on state gradients. Periodic âsyncâ: briefly return to true rollout gradients The lifted stop-gradient objective is great for fast, guided exploration, but itâs still an approximation of the original serial rollout objective. So every $K_{\text{sync}}$ iterations, GRASP does a short refinement phase: Roll out from $s_0$ using current actions $\mathbf{a}$, and take a few small gradient steps on the original serial loss: \[\mathbf{a} \leftarrow \mathbf{a} - \eta_{\text{sync}}\,\nabla_{\mathbf{a}}\,\|s_T(\mathbf{a})-g\|_2^2.\] The lifted-state optimization still provides the core of the optimization, while this refinement step adds some assistance to keep states and actions grounded towards real trajectories. This refinement step can of course be replaced with a serial planner of your choice (e.g. CEM); the core idea is to still get some of the benefit of the full-path synchronization of serial planners, while still mostly using the benefits of the lifted-state planning. How GRASP addresses long-range planning Collocation-based planners offer a natural fix for long-horizon planning, but this optimization is quite difficult through modern world models due to adversarial robustness issues. GRASP proposes a simple solution for a smoother collocation-based planner, alongside stable stochasticity for exploration. As a result, longer-horizon planning ends up not only succeeding more, but also finding such successes faster: Push-T demo: longer-horizon planning with GRASP. Horizon CEM GD LatCo GRASP H=40 61.4% / 35.3s 51.0% / 18.0s 15.0% / 598.0s 59.0% / 8.5s H=50 30.2% / 96.2s 37.6% / 76.3s 4.2% / 1114.7s 43.4% / 15.2s H=60 7.2% / 83.1s 16.4% / 146.5s 2.0% / 231.5s 26.2% / 49.1s H=70 7.8% / 156.1s 12.0% / 103.1s 0.0% / â 16.0% / 79.9s H=80 2.8% / 132.2s 6.4% / 161.3s 0.0% / â 10.4% / 58.9s Push-T results. Success rate (%) / median time to success. Bold = best in row. Note the median success time will bias higher with higher success rate; GRASP manages to be faster despite higher success rate. Whatâs next? There is still plenty of work to be done for modern world model planners. We want to exploit the gradient structure of learned world models, and collocation (lifted-state optimization) is a natural approach for long-horizon planning, but itâs crucial to understand typical gradient structure here: smooth and informative action gradients and brittle state gradients. We view GRASP as an initial iteration for such planners. Extension to diffusion-based world models (deeper latent timesteps can be viewed as smoothed versions of the world model itself), more sophisticated optimizers and noising strategies, and integrating GRASP into either a closed-loop system or RL policy learning for adaptive long-horizon planning are all natural and interesting next steps. I do genuinely think itâs an exciting time to be working on world model planners. Itâs a funny sweet spot where the background literature (planning and control overall) is incredibly mature and well-developed, but the current setting (pure planning optimization over modern, large-scale world models) is still heavily underexplored. But, once we figure out all the right ideas, world model planners will likely become as commonplace as RL. For more details, read the full paper or visit the project website. Citation @article{psenka2026grasp, title={Parallel Stochastic Gradient-Based Planning for World Models}, author={Michael Psenka and Michael Rabbat and Aditi Krishnapriyan and Yann LeCun and Amir Bar}, year={2026}, eprint={2602.00475}, archivePrefix={arXiv}, primaryClass={cs.LG}, url={https://arxiv.org/abs/2602.00475} }
Our latest image generation model offers advanced world knowledge, production ready specs, subject consistency and more, all at Flash speed.
Running advanced AI and computer vision workloads on small, power-efficient devices at the edge is a growing challenge. Robots, smart cameras, and autonomous...
On November 18, learn how to fine-tune the NVIDIA Cosmos Reason VLM with your own data to create visual AI agents.
Generative AI is opening new possibilities for analyzing existing video streams. Video analytics are evolving from counting objects to turning raw video content...
First unveiled at NVIDIA GTC 2025, NVIDIA Cosmos Reason is an open and fully customizable reasoning vision language model (VLM) for physical AI and robotics....
As enterprises generate and consume increasing volumes of diverse data, extracting insights from multimodal documents, like PDFs and presentations, has become a...
"In projecting language back as the model for thought, we lose sight of the tacit embodied understanding that undergirds our intelligence." –Terry WinogradThe recent successes of generative AI models have convinced some that AGI is imminent. While these models appear to capture the essence of human
Vision language models (VLMs) have transformed video analytics by enabling broader perception and richer contextual understanding compared to traditional...
This is the first post in the LLM Benchmarking series, which shows how to use GenAI-Perf to benchmark the Meta Llama 3 model when deployed with NVIDIA NIM. ...
PLAID is a multimodal generative model that simultaneously generates protein 1D sequence and 3D structure, by learning the latent space of protein folding models. The awarding of the 2024 Nobel Prize to AlphaFold2 marks an important moment of recognition for the of AI role in biology. What comes next after protein folding? In PLAID, we develop a method that learns to sample from the latent space of protein folding models to generate new proteins. It can accept compositional function and organism prompts, and can be trained on sequence databases, which are 2-4 orders of magnitude larger than structure databases. Unlike many previous protein structure generative models, PLAID addresses the multimodal co-generation problem setting: simultaneously generating both discrete sequence and continuous all-atom structural coordinates. From structure prediction to real-world drug design Though recent works demonstrate promise for the ability of diffusion models to generate proteins, there still exist limitations of previous models that make them impractical for real-world applications, such as: All-atom generation: Many existing generative models only produce the backbone atoms. To produce the all-atom structure and place the sidechain atoms, we need to know the sequence. This creates a multimodal generation problem that requires simultaneous generation of discrete and continuous modalities. Organism specificity: Proteins biologics intended for human use need to be humanized, to avoid being destroyed by the human immune system. Control specification: Drug discovery and putting it into the hands of patients is a complex process. How can we specify these complex constraints? For example, even after the biology is tackled, you might decide that tablets are easier to transport than vials, adding a new constraint on soluability. Generating âusefulâ proteins Simply generating proteins is not as useful as controlling the generation to get useful proteins. What might an interface for this look like? For inspiration, let's consider how we'd control image generation via compositional textual prompts (example from Liu et al., 2022). In PLAID, we mirror this interface for control specification. The ultimate goal is to control generation entirely via a textual interface, but here we consider compositional constraints for two axes as a proof-of-concept: function and organism: Learning the function-structure-sequence connection. PLAID learns the tetrahedral cysteine-Fe2+/Fe3+ coordination pattern often found in metalloproteins, while maintaining high sequence-level diversity. Training using sequence-only training data Another important aspect of the PLAID model is that we only require sequences to train the generative model! Generative models learn the data distribution defined by its training data, and sequence databases are considerably larger than structural ones, since sequences are much cheaper to obtain than experimental structure. Learning from a larger and broader database. The cost of obtaining protein sequences is much lower than experimentally characterizing structure, and sequence databases are 2-4 orders of magnitude larger than structural ones. How does it work? The reason that weâre able to train the generative model to generate structure by only using sequence data is by learning a diffusion model over the latent space of a protein folding model. Then, during inference, after sampling from this latent space of valid proteins, we can take frozen weights from the protein folding model to decode structure. Here, we use ESMFold, a successor to the AlphaFold2 model which replaces a retrieval step with a protein language model. Our method. During training, only sequences are needed to obtain the embedding; during inference, we can decode sequence and structure from the sampled embedding. âïž denotes frozen weights. In this way, we can use structural understanding information in the weights of pretrained protein folding models for the protein design task. This is analogous to how vision-language-action (VLA) models in robotics make use of priors contained in vision-language models (VLMs) trained on internet-scale data to supply perception and reasoning and understanding information. Compressing the latent space of protein folding models A small wrinkle with directly applying this method is that the latent space of ESMFold â indeed, the latent space of many transformer-based models â requires a lot of regularization. This space is also very large, so learning this embedding ends up mapping to high-resolution image synthesis. To address this, we also propose CHEAP (Compressed Hourglass Embedding Adaptations of Proteins), where we learn a compression model for the joint embedding of protein sequence and structure. Investigating the latent space. (A) When we visualize the mean value for each channel, some channels exhibit âmassive activationsâ. (B) If we start examining the top-3 activations compared to the median value (gray), we find that this happens over many layers. (C) Massive activations have also been observed for other transformer-based models. We find that this latent space is actually highly compressible. By doing a bit of mechanistic interpretability to better understand the base model that we are working with, we were able to create an all-atom protein generative model. Whatâs next? Though we examine the case of protein sequence and structure generation in this work, we can adapt this method to perform multi-modal generation for any modalities where there is a predictor from a more abundant modality to a less abundant one. As sequence-to-structure predictors for proteins are beginning to tackle increasingly complex systems (e.g. AlphaFold3 is also able to predict proteins in complex with nucleic acids and molecular ligands), itâs easy to imagine performing multimodal generation over more complex systems using the same method. If you are interested in collaborating to extend our method, or to test our method in the wet-lab, please reach out! Further links If youâve found our papers useful in your research, please consider using the following BibTeX for PLAID and CHEAP: @article{lu2024generating, title={Generating All-Atom Protein Structure from Sequence-Only Training Data}, author={Lu, Amy X and Yan, Wilson and Robinson, Sarah A and Yang, Kevin K and Gligorijevic, Vladimir and Cho, Kyunghyun and Bonneau, Richard and Abbeel, Pieter and Frey, Nathan}, journal={bioRxiv}, pages={2024--12}, year={2024}, publisher={Cold Spring Harbor Laboratory} } @article{lu2024tokenized, title={Tokenized and Continuous Embedding Compressions of Protein Sequence and Structure}, author={Lu, Amy X and Yan, Wilson and Yang, Kevin K and Gligorijevic, Vladimir and Cho, Kyunghyun and Abbeel, Pieter and Bonneau, Richard and Frey, Nathan}, journal={bioRxiv}, pages={2024--08}, year={2024}, publisher={Cold Spring Harbor Laboratory} } You can also checkout our preprints (PLAID, CHEAP) and codebases (PLAID, CHEAP). Some bonus protein generation fun! Additional function-prompted generations with PLAID. Unconditional generation with PLAID. Transmembrane proteins have hydrophobic residues at the core, where it is embedded within the fatty acid layer. These are consistently observed when prompting PLAID with transmembrane protein keywords. Additional examples of active site recapitulation based on function keyword prompting. Comparing samples between PLAID and all-atom baselines. PLAID samples have better diversity and captures the beta-strand pattern that has been more difficult for protein generative models to learn. Acknowledgements Thanks to Nathan Frey for detailed feedback on this article, and to co-authors across BAIR, Genentech, Microsoft Research, and New York University: Wilson Yan, Sarah A. Robinson, Simon Kelow, Kevin K. Yang, Vladimir Gligorijevic, Kyunghyun Cho, Richard Bonneau, Pieter Abbeel, and Nathan C. Frey.
Training Diffusion Models with Reinforcement Learning We deployed 100 reinforcement learning (RL)-controlled cars into rush-hour highway traffic to smooth congestion and reduce fuel consumption for everyone. Our goal is to tackle "stop-and-go" waves, those frustrating slowdowns and speedups that usually have no clear cause but lead to congestion and significant energy waste. To train efficient flow-smoothing controllers, we built fast, data-driven simulations that RL agents interact with, learning to maximize energy efficiency while maintaining throughput and operating safely around human drivers. Overall, a small proportion of well-controlled autonomous vehicles (AVs) is enough to significantly improve traffic flow and fuel efficiency for all drivers on the road. Moreover, the trained controllers are designed to be deployable on most modern vehicles, operating in a decentralized manner and relying on standard radar sensors. In our latest paper, we explore the challenges of deploying RL controllers on a large-scale, from simulation to the field, during this 100-car experiment. The challenges of phantom jams A stop-and-go wave moving backwards through highway traffic. If you drive, youâve surely experienced the frustration of stop-and-go waves, those seemingly inexplicable traffic slowdowns that appear out of nowhere and then suddenly clear up. These waves are often caused by small fluctuations in our driving behavior that get amplified through the flow of traffic. We naturally adjust our speed based on the vehicle in front of us. If the gap opens, we speed up to keep up. If they brake, we also slow down. But due to our nonzero reaction time, we might brake just a bit harder than the vehicle in front. The next driver behind us does the same, and this keeps amplifying. Over time, what started as an insignificant slowdown turns into a full stop further back in traffic. These waves move backward through the traffic stream, leading to significant drops in energy efficiency due to frequent accelerations, accompanied by increased CO2 emissions and accident risk. And this isnât an isolated phenomenon! These waves are ubiquitous on busy roads when the traffic density exceeds a critical threshold. So how can we address this problem? Traditional approaches like ramp metering and variable speed limits attempt to manage traffic flow, but they often require costly infrastructure and centralized coordination. A more scalable approach is to use AVs, which can dynamically adjust their driving behavior in real-time. However, simply inserting AVs among human drivers isnât enough: they must also drive in a smarter way that makes traffic better for everyone, which is where RL comes in. Fundamental diagram of traffic flow. The number of cars on the road (density) affects how much traffic is moving forward (flow). At low density, adding more cars increases flow because more vehicles can pass through. But beyond a critical threshold, cars start blocking each other, leading to congestion, where adding more cars actually slows down overall movement. Reinforcement learning for wave-smoothing AVs RL is a powerful control approach where an agent learns to maximize a reward signal through interactions with an environment. The agent collects experience through trial and error, learns from its mistakes, and improves over time. In our case, the environment is a mixed-autonomy traffic scenario, where AVs learn driving strategies to dampen stop-and-go waves and reduce fuel consumption for both themselves and nearby human-driven vehicles. Training these RL agents requires fast simulations with realistic traffic dynamics that can replicate highway stop-and-go behavior. To achieve this, we leveraged experimental data collected on Interstate 24 (I-24) near Nashville, Tennessee, and used it to build simulations where vehicles replay highway trajectories, creating unstable traffic that AVs driving behind them learn to smooth out. Simulation replaying a highway trajectory that exhibits several stop-and-go waves. We designed the AVs with deployment in mind, ensuring that they can operate using only basic sensor information about themselves and the vehicle in front. The observations consist of the AVâs speed, the speed of the leading vehicle, and the space gap between them. Given these inputs, the RL agent then prescribes either an instantaneous acceleration or a desired speed for the AV. The key advantage of using only these local measurements is that the RL controllers can be deployed on most modern vehicles in a decentralized way, without requiring additional infrastructure. Reward design The most challenging part is designing a reward function that, when maximized, aligns with the different objectives that we desire the AVs to achieve: Wave smoothing: Reduce stop-and-go oscillations. Energy efficiency: Lower fuel consumption for all vehicles, not just AVs. Safety: Ensure reasonable following distances and avoid abrupt braking. Driving comfort: Avoid aggressive accelerations and decelerations. Adherence to human driving norms: Ensure a ânormalâ driving behavior that doesnât make surrounding drivers uncomfortable. Balancing these objectives together is difficult, as suitable coefficients for each term must be found. For instance, if minimizing fuel consumption dominates the reward, RL AVs learn to come to a stop in the middle of the highway because that is energy optimal. To prevent this, we introduced dynamic minimum and maximum gap thresholds to ensure safe and reasonable behavior while optimizing fuel efficiency. We also penalized the fuel consumption of human-driven vehicles behind the AV to discourage it from learning a selfish behavior that optimizes energy savings for the AV at the expense of surrounding traffic. Overall, we aim to strike a balance between energy savings and having a reasonable and safe driving behavior. Simulation results Illustration of the dynamic minimum and maximum gap thresholds, within which the AV can operate freely to smooth traffic as efficiently as possible. The typical behavior learned by the AVs is to maintain slightly larger gaps than human drivers, allowing them to absorb upcoming, possibly abrupt, traffic slowdowns more effectively. In simulation, this approach resulted in significant fuel savings of up to 20% across all road users in the most congested scenarios, with fewer than 5% of AVs on the road. And these AVs donât have to be special vehicles! They can simply be standard consumer cars equipped with a smart adaptive cruise control (ACC), which is what we tested at scale. Smoothing behavior of RL AVs. Red: a human trajectory from the dataset. Blue: successive AVs in the platoon, where AV 1 is the closest behind the human trajectory. There is typically between 20 and 25 human vehicles between AVs. Each AV doesnât slow down as much or accelerate as fast as its leader, leading to decreasing wave amplitude over time and thus energy savings. 100 AV field test: deploying RL at scale Our 100 cars parked at our operational center during the experiment week. Given the promising simulation results, the natural next step was to bridge the gap from simulation to the highway. We took the trained RL controllers and deployed them on 100 vehicles on the I-24 during peak traffic hours over several days. This large-scale experiment, which we called the MegaVanderTest, is the largest mixed-autonomy traffic-smoothing experiment ever conducted. Before deploying RL controllers in the field, we trained and evaluated them extensively in simulation and validated them on the hardware. Overall, the steps towards deployment involved: Training in data-driven simulations: We used highway traffic data from I-24 to create a training environment with realistic wave dynamics, then validate the trained agentâs performance and robustness in a variety of new traffic scenarios. Deployment on hardware: After being validated in robotics software, the trained controller is uploaded onto the car and is able to control the set speed of the vehicle. We operate through the vehicleâs on-board cruise control, which acts as a lower-level safety controller. Modular control framework: One key challenge during the test was not having access to the leading vehicle information sensors. To overcome this, the RL controller was integrated into a hierarchical system, the MegaController, which combines a speed planner guide that accounts for downstream traffic conditions, with the RL controller as the final decision maker. Validation on hardware: The RL agents were designed to operate in an environment where most vehicles were human-driven, requiring robust policies that adapt to unpredictable behavior. We verify this by driving the RL-controlled vehicles on the road under careful human supervision, making changes to the control based on feedback. Each of the 100 cars is connected to a Raspberry Pi, on which the RL controller (a small neural network) is deployed. The RL controller directly controls the onboard adaptive cruise control (ACC) system, setting its speed and desired following distance. Once validated, the RL controllers were deployed on 100 cars and driven on I-24 during morning rush hour. Surrounding traffic was unaware of the experiment, ensuring unbiased driver behavior. Data was collected during the experiment from dozens of overhead cameras placed along the highway, which led to the extraction of millions of individual vehicle trajectories through a computer vision pipeline. Metrics computed on these trajectories indicate a trend of reduced fuel consumption around AVs, as expected from simulation results and previous smaller validation deployments. For instance, we can observe that the closer people are driving behind our AVs, the less fuel they appear to consume on average (which is calculated using a calibrated energy model): Average fuel consumption as a function of distance behind the nearest engaged RL-controlled AV in the downstream traffic. As human drivers get further away behind AVs, their average fuel consumption increases. Another way to measure the impact is to measure the variance of the speeds and accelerations: the lower the variance, the less amplitude the waves should have, which is what we observe from the field test data. Overall, although getting precise measurements from a large amount of camera video data is complicated, we observe a trend of 15 to 20% of energy savings around our controlled cars. Data points from all vehicles on the highway over a single day of the experiment, plotted in speed-acceleration space. The cluster to the left of the red line represents congestion, while the one on the right corresponds to free flow. We observe that the congestion cluster is smaller when AVs are present, as measured by computing the area of a soft convex envelope or by fitting a Gaussian kernel. Final thoughts The 100-car field operational test was decentralized, with no explicit cooperation or communication between AVs, reflective of current autonomy deployment, and bringing us one step closer to smoother, more energy-efficient highways. Yet, there is still vast potential for improvement. Scaling up simulations to be faster and more accurate with better human-driving models is crucial for bridging the simulation-to-reality gap. Equipping AVs with additional traffic data, whether through advanced sensors or centralized planning, could further improve the performance of the controllers. For instance, while multi-agent RL is promising for improving cooperative control strategies, it remains an open question how enabling explicit communication between AVs over 5G networks could further improve stability and further mitigate stop-and-go waves. Crucially, our controllers integrate seamlessly with existing adaptive cruise control (ACC) systems, making field deployment feasible at scale. The more vehicles equipped with smart traffic-smoothing control, the fewer waves weâll see on our roads, meaning less pollution and fuel savings for everyone! Many contributors took part in making the MegaVanderTest happen! The full list is available on the CIRCLES project page, along with more details about the project. Read more: [paper]
With the recent advancements in generative AI and vision foundational models, VLMs present a new wave of visual computing wherein the models are capable of...
Large language models (LLMs) have permeated every industry and changed the potential of technology. However, due to their massive size they are not practical...
Vision language models (VLMs) are evolving at a breakneck speed. In 2020, the first VLMs revolutionized the generative AI landscape by bringing visual...
Explore visually perceptive AI agents, the latest vision AI technologies, hands-on training, and inspiring deployments.
Empowering conservation efforts through innovative technologies and global collaborationA vessel captured by NASAâs Landsat 8. Skylightâs computer vision models leverage this imagery to identify suspicious behavior, such as illegal fishing, helping authorities act quickly to protect marine ecosystems.As our world faces unprecedented environmental challengesââârising global temperatures, increasingly severe extreme weather events, and a rapidly deepening biodiversity crisisâââthe need for innovative, scalable solutions has never been more urgent. AI is already playing a transformative role in addressing these global issues. For instance, AI camera traps equipped with satellite and cellular connectivity have reduced the data-to-action timeline from months to minutes, transforming how conservation is done. Meanwhile, advanced computer vision models are being used to detect and stop illegal fishing, estimate potential wildfire risks, and understand our changing planet. As a leader in developing state-of-the-art conservation technologies, Ai2 is heading to this yearâs United Nations Biodiversity Conference of the Parties (COP16) to showcase how open, collaborative AI can galvanize communities and solve the worldâs biggest problems.Taking place from October 21 to November 1 in Cali, Colombia, COP16 is one of the largest and most important global gatherings focused on addressing the urgent crisis of biodiversity loss. Current estimates indicate that at least one million plant and animal species are at risk of extinction, underscoring the need for immediate action. COP16 marks a pivotal moment as the first major opportunity to assess how national governments are moving from commitments to action.âWe are at a critical juncture where technology, collaboration, and urgency must intersect to drive real change for our planetââââAi2âs Ted SchmittWith 190 countries agreeing to protect 30 percent of the planetâs land and oceans by 2030, this yearâs conference will focus on the progress toward fulfilling that goal. Growing in just six years from 18 percent of land and 8 percent of the ocean currently protected, the need for tools to close this gap is urgent and critical. Ai2âs free and open-source technologies, like Skylight and EarthRanger, are uniquely positioned to empower governments and NGOs to turn their national action plans into reality by providing real-time data and insights to drive smarter, faster decisions in conservation efforts. This yearâs COP is a crucial event in assessing whether countries are on track and creating momentum to deliver on their promises and safeguard our planetâs biodiversity for future generations.A researcher points at her EarthRanger instance. Ai2âs EarthRanger is a cutting-edge platform designed for wildlife conservation, enabling real-time monitoring and data collection to protect threatened species and manage protected areas and landscapes effectively.At COP16, we will concentrate on two primary objectives: fostering and enhancing partnerships with global conservation stakeholders and inspiring recognition of the transformative potential of AI and conservation technologies in safeguarding biodiversity. While AI and technology are not a one-size-fits-all solution to the pressing challenges we face, they significantly accelerate the ability of conservationists and governments to adapt and respond to these rising threats. By co-hosting a pavilion in the Blue Zone with our partners, the Group on Earth Observations (GEO), GEO BON, and ESRI, we will demonstrate how nongovernmental organizations, collectively, are unlocking the data urgently needed to advance ecosystem protection and deliver on the ambitious targets set out in the global biodiversity framework.During COP16, we will host several presentations at the pavilion, including EarthRanger, Skylight, and our newly launched Earth System programs. We invited additional partners, including iNaturalist, leading AI for Conservation researchers at the Smithsonian Institution, MIT, WildLabs, and more, to share the latest technology developments in delivering conservation and environmental solutions. We will also join partners such as GEO, Maldivesâ Ministry of Climate Change, Environment, and Energy, Esri, Planet Labs, USGS, and Maxar to announce how the Global Ecosystems Atlas is empowering the Maldives to make better decisions faster. This initiative leverages advanced satellite imagery and AI to create the first-ever comprehensive map of ecosystems in the Maldives, providing a scalable, AI-powered resource that delivers detailed intelligence.In partnership with global spatial data providers, Ai2 will develop a globally accurate, regularly updated cropland map that will be immediately useful to a wide range of stakeholders seeking to better understand the vital food systems they manage. These AI-driven crop maps will strengthen agricultural monitoring to secure the availability of high-quality, nutritious food and economic stability.âWe are at a critical juncture where technology, collaboration, and urgency must intersect to drive real change for our planet,â said Ted Schmitt, Senior Director of Conservation at Ai2. âWe are excited to announce the launch of our latest philanthropic conservation technology program. EarthSystem will build on the highly impactful EarthRanger, Skylight, WildLands, and Climate programs. Through those programs, Ai2 is delivering real solutions to a range of environmental and biodiversity challenges. Leveraging our world-class AI researchers and engineers focused fully on conservation impact, EarthSystem will transform the power and promise of AI from dream to reality; we look forward to working with a range of partners across conservation and environmental domains to create game-changing solutions.âThese presentations will give attendees insights into how various technologies are helping to transform wildlife conservation, ocean health, and environmental monitoring. Presentations will take place throughout the conference, and we invite you to join us at the following sessions:Ai2 at COP16(all times Colombia Standard Time)Location- 22 October11:45â1 pm: Earth SystemâââHarnessing AI for Environmental Impact6:00â7:20 pm: Monitoring, control, and surveillance of future high seas MPAs: What role for emerging technologies (Skylight)- 23 October4:45â6:15 pm: Earth SystemâââHarnessing AI for Environmental Impact- 24 October11:40 am: Charting the FutureâââDeveloping Conservation Technology for Biodiversity Monitoring (EarthRanger and Ai2)3:00â4:30 pm: Transforming Wildlife Conservation with the Support of EarthRanger- 26 October1:15â2:45 pm: Transforming Wildlife Conservation with the Support of EarthRanger- 27 October1:15â2:45 pm: Earth SystemâââHarnessing AI for Environmental Impact6:30â7:30 pm: Transforming Wildlife Conservation with the Support of EarthRanger- 28 October3:00â4:30 pm: SkylightâââCombatting IUU Fishing with Cutting-Edge AI- 29 October11:45â1 pm: SkylightâââCombatting IUU Fishing with Cutting-Edge AI (presented in Spanish)- 30 October11:45â1 pm: SkylightâââCombatting IUU Fishing with Cutting-Edge AI1:15â2:45 pm: Earth SystemâââHarnessing AI for Environmental Impact6:00 pm: Mapping Natureâs Footprint: The Global Ecosystem AtlasAi2 at COP 16: Harnessing AI and Conservation Tech to Protect Our Planet was originally published in Ai2 Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.
The last few years of AI development have shown the power and potential of generative AI. Naturally, these leaps in machine intelligence have opened existential questions around AI safety. The fundamental question of how to build AI that benefits humanity while minimizing human harm remains largely unanswered. In fact, itâs hard to even know how to effectively approach the challenge of AI safety when we do not even have a unified definition of what constitutes safe AI.Why is AI safety so difficult to define? To start, safe or appropriate ways for a large language or multimodal model to respond are culturally dependent and vary wildly by semantics and context. âHow do you kill the lights in this room?â has a completely different meaning from âHow do you kill Casey in this room?â Humans can understand the difference, but naive, surface level AI safeguards would simply spot the word âkillâ and block a response.The starting point to developing safe AI models is, we must understand what the model understands. With closed models, where we are restricted to only the API output, we will never be able to truly learn what the model knows. Without an understanding of what the model knows, how itâs leveraging data to formulate a response, and what data is in the model, we have no hope of conducting the research that is required to design and effectively regulate AI models.The dynamics of making closed models safer is a mixed bag. Internal, largely undocumented research within large companies develops techniques to control model outputs. At the same time, researchers attempt to understand and âjailbreakâ the models, sharing the information freely with model providers. The integration of said feedback into closed models is undocumented and unofficial. Ultimately, solutions devised on top of closed models tend to act like band-aids that often cannot last the test of time, because they patch narrowly defined, specific behaviors one at a time. As the stakes of generative models raises, making this feedback loop of safety open is crucial to creating a healthy ecosystem.At Ai2 we are taking a long-term, scientific approach to creating models that are optimized for both model capability and safety. We believe the challenges around safety are bigger than what any individual institute could achieve alone, which is why we are empowering the whole community to conduct safety research by opening all of our data, models, and evaluations techniques end to end.Ai2 has developed principles to govern our approach to AI safety:AI safety is a technical problemWe need to first understand what the models understand. From there we can design safety into every stage of the AI development pipeline from data through to model training and evaluation, and with different audiences in mind.Our safety research focuses on the following areas:Enhancing the safety of OLMo, our open-source AI model, by exploring adding safety data into post-training pipelines.Improving the generation of large-scale synthetic training data for safety by investigating additional data mixtures and best practices.Identifying the root causes of harm in LLMs, as well as how to unlearn harmful behavior while enforcing safety and preserving general capabilities.Discovering novel failure cases by improving methods for large-scale automatic jailbreaking.Expanding safety evaluation benchmarks to detect and understand both known and unknown risks.Sharing our learnings to advance safety practices across academia and industry.Safety research needs to be done in the openIn order to create a safe language model, safety needs to be involved in every step of the training process. To enable the entire research community to conduct safety research, Ai2 makes all elements of the AI development pipeline free and openly available via permissive licenses:OLMo, pre-training data, training code, logs, and checkpoints make an entire framework that enables researchers and developers to iterate work without starting from scratch or remaining âin the darkâ about model details.Dolma is an explorable dataset, so users can see the finer details of what is going into the data that trains models.Our partnership with DSRI will further aim to bring important safety activities like red teaming to light for the larger community.Designing for safety is a continuous processSafety is an ongoing research problem. We are constantly learning the community expectations for how to build and deploy a safe language model. For example, when should models not comply with user inquiries? Not only when the inquiries are clearly dangerous, but also when a user is asking for information a model cannotâââor should notâââprovide. CoCoNot is a dataset crafted to help models train on situations when they should not comply with user requests, and is another important piece of the safety puzzle.Safe AI needs to be multilingual and culturally awareSafety must go beyond Euro-centric standards, but much of the existing toxicity research done on LLMs is in English. We aim to open up the conversation by including more people, and languages, in our research. PolygloToxicityPrompts is a dataset of 425K naturally-occurring prompts across 17 languages with varying degrees of toxicity, and only scratches the surface of inclusivity we strive for.The Ai2 Safety Toolkit: A Hub for Open CollaborationRecently, we introduced the Ai2 Safety Toolkit, a central hub for advancing LLM safety and fostering open science. It is a suite of resources focused on LLM safety that empowers researchers and industry professionals to collaborate on building safer AI models.As a first step towards this effort, we rolled out two major components that offer content safety moderation tools and a mechanism to protect against in-the-wild attacks.WildTeaming, an automatic red-teaming framework for identifying and reproducing human-devised attacks. It can identify up to 4.5 times more unique successful attacks than prior systems. It also enables the creation of the WildJailbreak, a high-quality, large-scale safety training dataset with 262K training examples, which substantially updates prior public safety training resources.WildGuard, a light-weight, multi-purpose moderation tool for assessing the safety of user-LLM interactions across three safety moderation tasks including prompt harmfulness, response harmfulness, and response refusal. It includes WildGuardMix, a carefully balanced, multi-task moderation dataset with 92K labeled examples covering 13 risk categoriesâââthe largest multi-task open safety dataset to date.To ensure the integrity and effectiveness of our safety research, we have established a Safety Committee and a Safety Review Board. These bodies review research at Ai2 to ensure our work meets the highest bar for ethical, human-centered AI research and development. This work is closely aligned with our work in NAIRR and our other public and private research partners.Continuing to safeguard AIThe safe development of AI is a continuous process, there is no simple safe or unsafe implementation. Researchers, policy-makers, large enterprises, and all of society must collectively support ongoing research and work together to ensure AI serves humanity in a safe, open, responsible, and ethical way.ResourcesLearn about the Ai2 Safety ToolkitSafeguard your model with WildGuardJoin us for red teaming OLMo at DEF CON on August 12â13 in Las VegasFollow @allen_ai on Twitter/X, and subscribe to the Ai2 Newsletter to stay current on news and research coming out of Ai2.Open research is the key to unlocking safer AI was originally published in Ai2 Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.