RESEARCH

AI PAPERS

Stay updated with the latest AI research papers from leading institutions and labs around the world.

FEATURED PAPERS

Attension is all you need

The dominant sequence transduction models are based on complex recurrent or convolutional neural networks that include an encoder and a decoder. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 Englishto-German translation task, improving over the existing best results, including ensembles, by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.0 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature.

Google•6/4/2026

Categories:

llm

paper

NLP

LLMs

Deep Unsupervised Learning

What’s in GPT-5

Alan D. Thompson27 Jul 2026

https://lifearchitect.ai

<h1>What’s in GPT-5? A Deep Dive into the Data Powering Next-Generation AI</h1>As large language models grow more powerful, one thing becomes increasingly scarce: transparency. OpenAI, like many frontier AI labs, has become notably quiet about the specific data used to train its flagship models. But that hasn't stopped researchers from piecing together the puzzle.In August 2024, Alan D. Thompson of <a target="_blank" rel="nofollow noopener noreferrer" class="text-accent-mint underline" href="http://LifeArchitect.ai">LifeArchitect.ai</a> released a comprehensive 27-page report titled "What's in GPT-5?" — a meticulous investigation into the datasets likely used to train OpenAI's next-generation model. The report has been reviewed by all major AI labs and intergovernmental organizations, and was even cited in the 2024 G7 AI document provided to world leaders including President Biden, Prime Minister Starmer, President Macron, and othersThe investigation reveals that GPT-5’s raw pre-training corpus approaches approximately 500 trillion tokens (2 petabytes), condensing to over 70 trillion tokens (281 terabytes) after filtering—representing a dramatic scale increase over its predecessors. The dataset draws from over 27 distinct sources, categorized across web data (Common Crawl, Reddit), commercial news partnerships (News Corp, Associated Press, Financial Times), books (Books1–3), academic literature (Wikipedia, Consensus NLP), legal documents (FreeLaw), code repositories (GitHub, Stack Exchange), and multilingual resources. Critically, the analysis documents a paradigm shift toward synthetic data generation, tracing its evolution from Microsoft’s 1-billion-token experiments in late 2023 to Hugging Face’s 25-billion-token Cosmopedia pipeline in early 2024. For GPT-5, synthetic content—generated across 110 curated topics and audience types—is projected to constitute a substantial fraction of the final training mix. The findings highlight a broader industry trend of diminishing transparency, while quantitatively establishing the foundational data infrastructure required for next-generation frontier models. This work provides essential empirical grounding for assessing GPT-5’s emergent capabilities, biases, and the strategic implications of synthetic data as a compounding asset in AI development.

llm

paper

View

TRIBE v2: A Predictive Foundation Model Trained to Understand How the Human Brain Processes Complex Stimuli

Stephane d'Ascoli, Jeremy Rapin, Yohann Benchetrit +57 Jun 2026

Attension is all you need

Llion Jones4 Jun 2026

Google

View

The AI Scientist-v2

Yutaro, Robert, etc28 Feb 2024

sakana ai

AI is increasingly playing a pivotal role in transforming how scientific discoveries are made. We introduce The A I Sc ient ist -v2, an end-to-end agentic system capable of producing the first entirely AIgenerated peer-review-accepted workshop paper. This system iteratively formulates scientific hypotheses, designs and executes experiments, analyzes and visualizes data, and autonomously authors scientific manuscripts. Compared to its predecessor (v1, Lu et al., 2024), The A I Sc ient ist -v2 eliminates the reliance on human-authored code templates, generalizes effectively across diverse machine learning domains, and leverages a novel progressive agentic tree-search methodology managed by a dedicated experiment manager agent. Additionally, we enhance the AI reviewer component by integrating a Vision-Language Model (VLM) feedback loop for iterative refinement of content and aesthetics of the figures. We evaluated The A I Sc ient ist -v2 by submitting three fully autonomous manuscripts to a peer-reviewed ICLR workshop. Notably, one manuscript achieved high enough scores to exceed the average human acceptance threshold, marking the first instance of a fully AI-generated paper successfully navigating a peer review. This accomplishment highlights the growing capability of AI in conducting all aspects of scientific research. We anticipate that further advancements in autonomous scientific discovery technologies will profoundly impact human knowledge generation, enabling unprecedented scalability in research productivity and significantly accelerating scientific breakthroughs, greatly benefiting society at large. We have open-sourced the code at <a target="_blank" rel="noopener noreferrer nofollow" class="text-accent-mint underline" href="https://github.com/SakanaAI/AI-Scientist-v2">https://github.com/SakanaAI/AI-Scientist-v2</a> to foster the future development of this transformative technology. We also discuss the role of AI in science, including AI safety.

LLMs

View

Deep Unsupervised Learning

Jascha Sohl-Dickstein, Eric A. Weiss, Niru Maheswaranathan +118 Nov 2015

https://arxiv.org

A central problem in machine learning involves modeling complex data-sets using highly flexible families of probability distributions in which learning, sampling, inference, and evaluation are still analytically or computationally tractable. Here, we develop an approach that simultaneously achieves both flexibility and tractability. The essential idea, inspired by non-equilibrium statistical physics, is to systematically and slowly destroy structure in a data distribution through an iterative forward diffusion process. We then learn a reverse diffusion process that restores structure in data, yielding a highly flexible and tractable generative model of the data. This approach allows us to rapidly learn, sample from, and evaluate probabilities in deep generative models with thousands of layers or time steps, as well as to compute conditional and posterior probabilities under the learned model. We additionally release an open source reference implementation of the algorithm.

Deep Unsupervised Learning

View

About AI Papers on TRONLAB

The AI research landscape moves fast. New papers drop daily from labs at Google, Meta, OpenAI, Anthropic, and universities worldwide — each potentially reshaping how we work with AI. We curate the most impactful publications and present them in an accessible format so you can stay informed without spending hours on arXiv.

Papers are categorised by research area (NLP, Computer Vision, Reinforcement Learning, LLMs, Multi-Modal AI, etc.) and tagged with key concepts. Featured papers are hand-picked for their practical relevance to AI practitioners — not just academic significance. Whether you're a developer wanting to understand the latest architecture innovations, or a product manager tracking capability frontiers, this is your curated reading list.