The week's developments in AI, quantum, and biotech, explained | 05.06.24

If you’re enjoying this briefing, sign up here — and share it with a friend. Reading time: ~4 minutes

Data is essential in the age of AI, serving as the lifeblood of complex LLMs. However, AI's insatiable demand for information poses significant challenges. 

Unleashing AI's true power requires seamless data integration. Hitachi Vantara, a leader in AI data infrastructure, is tackling this for enterprises. We spoke with Jason Hardy, CTO for AI at Hitachi Vantara, about their role in projects like the Las Vegas Sphere and the future of data processing in healthcare, finance, sustainability, and more. 

Read on for a short excerpt below, or cut to the chase and read the full piece on the CogX Blog.

We cover this, plus an AI startup that is literally trash, the new definition of AGI, and the Swiss AI startup saving chocolate one algorithm at a time (and much more).

— Charlie and the Research & Intelligence Team 

P.S Exciting News! We're giving away a FREE pair of tickets to LA 2025! To enter, simply share your best photos from #CogXUSA and tag us using our social media handles (Insta: @cogxusa, Twitter: @Cogx_festival, LinkedIn: CogX Festival). Good luck!

Solving the AI Data Challenge: How Hitachi Vantara's Technology is Enabling the Next Generation of AI

“Solving the [AI] data challenge is at the core of who Hitachi Vantara is.”

1. What was Hitachi Vantara’s role in the Sphere project? What did you hope to achieve?

Hitachi Vantara was inspired by Sphere’s vision for a new style of immersive entertainment at a scale and quality unlike anything else. Our goal was to help make sure that the technology behind it was ready to meet the challenge. With the world’s highest resolution LED screen inside the venue and the largest LED screen on the exterior, Sphere leaned on Hitachi Vantara to deliver solutions to stream video content on an unprecedented scope.

Hitachi Vantara worked in tandem with Sphere’s engineering teams and long-time integrator, 7thSense, to evolve the high-speed storage capabilities provided by Hitachi Content Software for File (HCSF) to predictably and reliably maintain the extremely high-performance requirements necessary. For Sphere’s original immersive film, Darren Aronofsky’s Postcard From Earth, the system handles over 400 gigabytes a second of throughput at sub 5 milliseconds of latency and a 12-bit color display at a 444 subsampling. The HCSF system consists of 27 nodes, with 4PB of flash storage for playback within Sphere and streamed in real-time to 7thSense media servers, each streaming 4K video at 60 frames per second – a world-first in terms of technology capability at this scale.

This mission critical requirement demanded nothing but success, and I am proud to say that our collaboration more than delivered. Achieving a smooth video playback experience at this scale demanded coordination between all aspects of the pipeline. HCSF, being integral to this experience, ensured that 7thSense’s software was able to deliver to the screen in a consistent manner.

2. How did Hitachi Vantara handle and process the enormous amount of data required for this project? What technologies were crucial in achieving this? 

Hitachi Content Software for File is the same technology that other Hitachi Vantara customers utilize for massive AI and Generative AI requirements – including foundational model training – as well as massive data lake house requirements, high speed data analytics, genome research, and other mission critical workloads that operate at a massive scale. We determined that HCSF would be capable of managing the scale and raw performance requirements of this project because of its history of success in other critical environments. The challenge that Sphere brought included the density requirements for a system of this scale, and the latent-sensitive environment we were working in. Being able to serve the video streaming requirements to Sphere took some very creative engineering to ensure the delivery was consistent and the performance and latency were always within the tolerances. 

… want to keep reading? Check out the full interview here on the CogX Blog.

Share your expertise! Want to be a guest contributor in our next issue? drop us a line at: editors@cogx.live 

Innovation and Releases

♻️This AI is rubbish, literally. Finnish startup Binit is launching an AI gadget to monitor household trash. With integrated sensors and cameras, users can scan items before disposal, receiving analytics and feedback, leading to a 40% reduction in mixed bin waste.

🧠Microsoft commits $3.2B to Sweden for AI expansion, deploying 20,000 advanced GPUs in data centres to boost competitiveness, as the country currently lags behind its Nordic neighbours in AI adoption. Over three years, they’ll train 250,000 people in AI skills. 

🍫Swiss AI startup saving chocolate secures $16M: EthonAI uses AI tech to reduce factory waste, optimise production and maintain quality through machine fine-tuning and sensor data monitoring, Customers include Siemens, Roche, and Lindt & Spruengli. 

🤖Industries ready for robots, are robots ready for them? Despite the robotics hype, and investors anticipating market growth, humanoid robots are rarely seen at industry events. Pilots and partnerships hint at progress, yet market fit still remains unclear. 

🍷Hugging Face releases groundbreaking dataset for LLM pretraining. FineWeb offers 15 trillion tokens, occupying 44TB. Deduplication using MinHash enhances model performance, plus its advanced filtering ensures high-quality content. 

AI Tools of the Week

🖋️Improve your font pairings: Fontjoy uses deep learning to create balanced and aesthetically pleasing font combinations. 

🧠The future of search? IKI AI interprets user queries and provides concise answers from diverse data sources, reducing retrieval time and enhancing task efficiency.

💡Crypto whitepapers, explained: ChainClarity uses AI to break down complex cryptocurrency whitepapers into clear, concise summaries.

Latest Research

🧬Novel Software decodes disease mechanisms: Researchers have developed "STew," a machine learning method that combine gene activity with tissue location to identify  patterns in diseased tissues, enhancing understanding and guiding better treatments.

👍Public easily adapts to an extra thumb, research finds. But why? An extra, robotic thumb is shown to increase productivity, aid object manipulation, and help those with disabilities better interact with their environment. Good news it's easy to get to grips with. 

📊AI moves deeper into scientific research, as hallucinations wane: Norwegian startup Iris secured €7.6M in funding for its ML engine that analyses academic research, providing accurate insights by cross-checking generated content against knowledge bases.

🧠AGI = Artificial Generational Intelligence? Unlike traditional reinforcement learning, this new approach balances social and independent learning across generations. The result? AI agents that continually build and refine knowledge, outperforming isolated training methods.

In case you missed it

Listen to Roman Yampolskiy, AI safety researcher and author, chat with Lex Fridman, about the dangers of Superintelligent AI.

✍️ Enjoying this newsletter?  Subscribe to our free weekly briefings on Preparing for AI, Cinema & AI, The Race to Net Zero, AI & DeepTech and The Future of Work.

🚀 You are now able to sign up for the super early bird offer (75% off) to the CogX AI Summit in London on October 7th here.