No prompt words required, Stability AI demonstrates MindEye: the target can generate whatever it wants

News on March 21st, the AI ​​wave is sweeping across. Previously, many people thought that "prompt word engineer" would become an emerging job type, and the advent of MindEye shows that This position may have no existing value.

Many people have believed that the key to the AI ​​era is not the power of the models themselves, but rather whether humans can effectively use these AI models to complete specific tasks.

Therefore, the concept of "prompt word engineers" was proposed. They have deeper understanding capabilities and can provide more accurate prompt words for artificial intelligence, helping AI better meet user needs.

StabilityAI launched MindEye1 in July 2023, and recently launched MindEye2 again, greatly reducing the value of the "prompt word engineer". This model does not rely on specific prompt words, but is directly based on the user's brain. Radio wave generation, that is to say, whatever the user’s brain wants, the model will be able to generate in the future.

MindEye reconstructs and retrieves images directly from fMRI brain activity and can convert 2D images into 3D videos.

Functional magnetic resonance imaging (fMRI) is a neuroimaging technique that measures brain activity by detecting changes in blood flow to the brain. The main purpose of this technology is to map the brain's functions, providing important data for studying brain activity and evaluating potential treatments for neurological diseases. Through fMRI technology, we can gain an in-depth understanding of the brain's activity patterns when performing different tasks, thereby helping scientists better understand how the brain works.

MindEye is an MRI-based A system for observing data sets of participants' brain activity on an imaging scanner. The research team used these recordings to train a system capable of analyzing and retrieving raw images or generating reconstructed images. The system is able to reconstruct the images participants saw from their brain activity as they viewed a series of static images, providing researchers with valuable insights and information. In this way, MindEye can help researchers gain a deeper understanding of the brain's activity patterns and mechanisms during visual processing. This technology holds promise for neuroscience research

Researchers demonstrated that MindEye outperformed previous methods in image retrieval tasks, identifying original images from candidate images with over 90% accuracy. For reconstruction, MindEye uses pre-trained generative models.

MindEye can be applied in various fields. In the medical field, its ability to reconstruct visual perception from brain activity could be used in diagnostic and assessment methods, especially in situations where patients have difficulty communicating. MindEye’s real-time analytics potential promises to improve the performance of brain-computer interfaces.

The research team highlighted limitations related to data collection, including the lengthy scanning times required and the potential for data noise due to participant movement or inattention.

This site attaches paper references

• Reconstructing the Mind's Eye: fMRI-to-Image with Contrastive Learning and Diffusion Priors
• MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data
• ##MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data