TL;DR
Get business pricing on tech for your team
- Business-only prices and quantity discounts
- Tax-exempt purchasing
- Multiple users, one account, clear invoices
Import AI issue 476 reviews three developments: Humanity’s Sixth Sense, a benchmark where leading AI models scored well below people; TasteVal, a test of AI research judgment; and an analysis of how AI could speed up space industrialization. The supplied report gives detailed results for the visual benchmark, but its TasteVal account ends before reporting scores or conclusions.
Import AI issue 476 examines three developments in artificial intelligence: a benchmark showing a wide gap between people and models on intuitive visual reasoning, a new evaluation of AI research judgment, and an analysis of how autonomous AI could accelerate work in space. The issue’s most fully reported results come from Humanity’s Sixth Sense, where the leading system described scored 53.6, compared with 93.1 for the human baseline.
Elorian and Scale AI developed Humanity’s Sixth Sense (HSS) to test whether models can infer information implied by images and video rather than explicitly shown. The benchmark contains 522 questions based on 288 images and 234 video clips, which together total 17.6 hours. Questions cover temporal and causal dynamics, physical and spatial reasoning, social understanding, and abstract or contextual inference.
In the results summarized by Import AI, people achieved an aggregate pass@1 score of 93.1. The best-scoring system listed, GPT 6 Astra, scored 53.6. Its reported scores were 55.6 on image questions, 51.1 on video, 58.2 on time-related questions and 53.6 on space-related questions. The issue also reports scores of 26.8 for GLM-5.3-flash and 25.5 for Kimi K3, described there as Chinese open-weight models.
The report says the most frequent individual errors were missing a detail that determined the answer, accounting for 21% of failures, and misidentifying an object, person or role, accounting for 20%. It also introduces TasteVal, a P-Zero Research benchmark covering eight tasks associated with frontier AI development, including data curation, model training, preference modeling and resistance to adversarial prompts. The supplied text does not include TasteVal results. A third section discusses an AI Frontiers article arguing that autonomous spacecraft, space-based infrastructure and manufacturing could give AI a larger role beyond Earth.
Where Visual Models Still Fall Short
The HSS results offer a concrete measure of a capability often described broadly as intuitive visual reasoning. The gap between the reported human and leading-model scores suggests that strong performance on selected image or video tasks does not yet amount to human-level performance across this benchmark. The reported error types also point to practical weaknesses: overlooking a decisive detail or assigning the wrong identity or role can change an answer even when a scene is broadly understood.
At the same time, Import AI’s interpretation is more qualified than a simple account of failure. It notes that language-based multimodal systems performed on tasks that some researchers have argued would require dedicated spatial world models. That observation may inform debate about how visual reasoning can be achieved, but this benchmark alone does not establish that such models are unnecessary or settle how systems reason internally.
The space analysis addresses a different concern: if spacecraft and infrastructure become more autonomous, AI could affect not only research and production but also the pace and distribution of activity in orbit and beyond. The article presents these outcomes as possibilities, not as a report that such an AI-driven space economy already exists.
AI visual reasoning benchmark test kit
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Three Threads in Issue 476
Import AI is a newsletter about AI research. Issue 476 brings together a benchmark report, a research-evaluation project and an argument about longer-term space activity; these are related by their focus on what AI systems may be able to do, but they are distinct developments rather than one coordinated announcement.
HSS asks about information that a scene implies but does not state directly. Its examples include judging whether a bookcase can hold more books or whether chains shown in a scene can bear the force of a plough. Its categories span visual, temporal, spatial and social inference. The newsletter notes that the human baseline is high but not perfect, so the comparison is between measured performance on this test, not a claim that people answer every question correctly.
TasteVal is presented as an evaluation of research taste: the ability to choose promising directions or approaches across tasks involved in developing frontier AI systems. Separately, the AI Frontiers article considers how autonomous spacecraft and space-based energy, computation and manufacturing might develop. Its discussion includes possible advantages for early entrants, but the material supplied does not report a project, launch or policy decision implementing those scenarios.
“Measures how a model interprets and reasons through information a visual scene implies but doesn’t explicitly show.”
— Elorian and Scale AI, describing Humanity’s Sixth Sense
AI research judgment evaluation tools
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Limits of the Reported Evidence
The supplied account does not give the HSS paper’s full methodology, model-testing conditions, uncertainty estimates or complete list of systems. The scores should be read as results reported for this benchmark, not as a general ranking of all visual AI capabilities. The excerpt also does not specify the evaluation date or model versions beyond the names provided.
For TasteVal, the source text cuts off while describing what the benchmark is intended to measure. It provides no model scores, comparison baseline or conclusions, so claims about which systems show better research judgment cannot be verified from this material. The space section likewise outlines potential mechanisms and risks, including compounding infrastructure and strategic advantages, but gives no timetable or evidence that those scenarios are underway.
space industrialization AI automation
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Results Needed to Test the Claims
The next useful evidence for HSS would include the full paper’s evaluation details, additional model results and tests showing how performance changes across task types and settings. The benchmark developers’ paper and introduction are cited by Import AI, but the supplied source does not state a release schedule for further results.
For TasteVal, readers would need the complete benchmark report and its results to assess whether models can reliably make useful research choices, and how those choices compare with human researchers or other systems. On space, the issues raised by AI Frontiers will depend on developments in launch capacity, autonomous spacecraft and space-based infrastructure; the newsletter excerpt identifies no specific upcoming milestone or decision.
As an affiliate, we earn on qualifying purchases.
Key Questions
What is Humanity’s Sixth Sense?
Humanity’s Sixth Sense is a benchmark by Elorian and Scale AI that tests whether models can infer information implied by images and video. It includes 522 questions based on 288 images and 234 video clips.
How did the leading model compare with people?
In the results reported by Import AI, the human baseline scored 93.1 in aggregate, while GPT 6 Astra scored 53.6. These are scores on this benchmark, not a general measure of every visual reasoning task.
What does the report say about TasteVal?
It says P-Zero Research developed TasteVal to evaluate research judgment across eight tasks related to frontier AI development. The supplied text does not include scores or findings, so it does not establish how well any model performed.
Is AI already directing a space economy?
The space section discusses possible future effects of autonomous spacecraft and space-based infrastructure. The source presents these as scenarios and risks, not as confirmation that an AI-led space economy is already operating.
Source: rss
Halloween Picks
halloween
As an affiliate, we earn on qualifying purchases.
