This video is organized as follows: the movie that each subject viewed while in the magnet is shown at upper left. Reconstructions for three subjects are shown in the three rows at bottom. All these reconstructions were obtained using only each subject's brain activity and a library of 18 million seconds of random YouTube video that did not include the movies used as stimuli. (In brief, the algorithm processes each of the 18 million clips through the brain model, and identifies the clips that would have produced brain activity as similar to the measured activity as possible. The clips used to fit the model, the clips used to test the model and the clips used to reconstruct the stimulus were entirely separate.) The reconstruction at far left is the Average High Posterior (AHP). The reconstruction in the second column is the Maximum a Posteriori (MAP). The other columns represent less likely reconstructions. The AHP is obtained by simply averaging over the 100 most likely movies in the reconstruction library. These reconstructions show that the process is very consistent, though the quality of the reconstructions does depend somewhat on the quality of brain activity data recorded from each subject.

The left clip is a segment of a Hollywood movie trailer that the subject viewed while in the magnet. The right clip shows the reconstruction of this segment from brain activity measured using fMRI. The procedure is as follows:

[1] Record brain activity while the subject watches several hours of movie trailers.

[2] Build dictionaries (i.e., regression models) that translate between the shapes, edges and motion in the movies and measured brain activity. A separate dictionary is constructed for each of several thousand points at which brain activity was measured.

(For experts: The real advance of this study was the construction of a movie-to-brain activity encoding model that accurately predicts brain activity evoked by arbitrary novel movies.)

[3] Record brain activity to a new set of movie trailers that will be used to test the quality of the dictionaries and reconstructions.

[4] Build a random library of ~18,000,000 seconds (5000 hours) of video downloaded at random from YouTube. (Note these videos have no overlap with the movies that subjects saw in the magnet). Put each of these clips through the dictionaries to generate predictions of brain activity. Select the 100 clips whose predicted activity is most similar to the observed brain activity. Average these clips together. This is the reconstruction.

Nikolaj ha scritto:

si quoto

L'attività cerebrale è registrata dalla risonocanza magnetica come una sorta di mappa tridimensionale del cervello, in cui ogni cubetto di pochi millimetri di lato,un cosidetto voxel,è associato al corrispettivo livello di attivazione dei neuroni al suo inyerno,indicato da quanto sangue li sta alimnetando.I ricercartori hanno costruito una serir di mappe di attività cerebrale,corrispondenti ognuna a una scena diversa.La presenza di una figura umana,per esempio attivava una certa mappa di voxel,diversa da quella di un treno in corsa.

Accumulati molti dati,Nishimoto e Gallant hanno poi tentato di far ricostruire a un software cosa stesse vedendo un volontario sulla base dell'attivazione dei voxel nella sua risononanza magnetica. Il software ha fuso l'insieme di scene casuali tratte da Youtube,fino a ricrearne una che,secondo i suoi calcoli, avrebbe prodotto la stessa attività cerebrale registrata nella persona in quel momento.

L'immagine è confusa,ma in effetti simile a quella vista.La speranza è che in futuro si possano visualizzare su uno schermo i pensieri di persone che non possono comunicare,perchè paralizzate o in stato vegetativo.

Fonte:La Repubblica