Scientists Reconstruct Brains’ Visions Into Digital Video In Historic Experiment

• Conor wrote on 2011-09-26 01:03

  
  UC Berkeley scientists have developed a system to capture visual activity in human brains and reconstruct it as digital video clips. Eventually, this process will allow you to record and reconstruct your own dreams on a computer screen.
  
  I just can't believe this is happening for real, but according to Professor Jack Gallant—UC Berkeley neuroscientist and coauthor of the research published today in the journal Current Biology—"this is a major leap toward reconstructing internal imagery. We are opening a window into the movies in our minds."
  
  Indeed, it's mindblowing. I'm simultaneously excited and terrified. This is how it works:
  They used three different subjects for the experiments—incidentally, they were part of the research team because it requires being inside a functional Magnetic Resonance Imaging system for hours at a time. The subjects were exposed to two different groups of Hollywood movie trailers as the fMRI system recorded the brain's blood flow through their brains' visual cortex.
  
  The readings were fed into a computer program in which they were divided into three-dimensional pixels units called voxels (volumetric pixels). This process effectively decodes the brain signals generated by moving pictures, connecting the shape and motion information from the movies to specific brain actions. As the sessions progressed, the computer learned more and more about how the visual activity presented on the screen corresponded to the brain activity.
  
  [SIZE="3"]An 18-million-second picture palette[/SIZE]
  
  After recording this information, another group of clips was used to reconstruct the videos shown to the subjects. The computer analyzed 18 million seconds of random YouTube video, building a database of potential brain activity for each clip. From all these videos, the software picked the one hundred clips that caused a brain activity more similar to the ones the subject watched, combining them into one final movie. Although the resulting video is low resolution and blurry, it clearly matched the actual clips watched by the subjects.
  
  Think about those 18 million seconds of random videos as a painter's color palette. A painter sees a red rose in real life and tries to reproduce the color using the different kinds of reds available in his palette, combining them to match what he's seeing. The software is the painter and the 18 million seconds of random video is its color palette. It analyzes how the brain reacts to certain stimuli, compares it to the brain reactions to the 18-million-second palette, and picks what more closely matches those brain reactions. Then it combines the clips into a new one that duplicates what the subject was seeing. Notice that the 18 million seconds of motion video are not what the subject is seeing. They are random bits used just to compose the brain image.
  
  Given a big enough database of video material and enough computing power, the system would be able to re-create any images in your brain.
  
  
  In this other video you can see how this process worked in the three experimental targets. On the top left square you can see the movie the subjects were watching while they were in the fMRI machine. Right below you can see the movie "extracted" from their brain activity. It shows that this technique gives consistent results independent of what's being watched—or who's watching. The three lines of clips next to the left column show the random movies that the computer program used to reconstruct the visual information.
  
  Right now, the resulting quality is not good, but the potential is enormous. Lead research author—and one of the lab test bunnies—Shinji Nishimoto thinks this is the first step to tap directly into what our brain sees and imagines:
  
  "Our natural visual experience is like watching a movie. In order for this technology to have wide applicability, we must understand how the brain processes these dynamic visual experiences."
  
  [SIZE="3"]The brain recorders of the future[/SIZE]
  
  Imagine that. Capturing your visual memories, your dreams, the wild ramblings of your imagination into a video that you and others can watch with your own eyes.
  
  This is the first time in history that we have been able to decode brain activity and reconstruct motion pictures in a computer screen. The path that this research opens boggles the mind. It reminds me of Brainstorm, the cult movie in which a group of scientists lead by Christopher Walken develops a machine capable of recording the five senses of a human being and then play them back into the brain itself.
  
  This new development brings us closer to that goal which, I have no doubt, will happen at one point. Given the exponential increase in computing power and our understanding of human biology, I think this will arrive sooner than most mortals expect. Perhaps one day you would be able to go to sleep wearing a flexible band labeled Sony Dreamcam around your skull. [UC Berkeley]

  
  Source: http://gizmodo.com/5843117/scientists-reconstruct-video-clips-from-brain-activity
  waaat..
  Honestly this freaks me out a little l0l
• Athde wrote on 2011-09-26 01:11
  GREAAAAAAT, like the dreams weren't bad enough, now I can save them and sell to Hollywood for Horror movie material...
  
  
  No but this is really awesome :D
  All they're missing now is sound and then it'll become a family past-time lol
• TA wrote on 2011-09-26 01:17
  omg.... I could make some kickass [s]porn[/s] movies!
  
  I always have dreams that seem like they would be THE best movies ever, but I can never fully capture them in writing.
• Athde wrote on 2011-09-26 01:21

  Quote from TA;599903:

  omg.... I could make some kickass [s]porn[/s] movies!
  
  I always have dreams that seem like they would be THE best movies ever, but I can never fully capture them in writing.

  
  
  ...that deserves this....
  

  [Image: http://s3.amazonaws.com/kym-assets/photos/images/original/000/137/570/50a21ab196c6c53c659796ab167d76b3.jpg?1308603083]
• NomadTrooper wrote on 2011-09-26 02:02
  I see this only being used for nefarious purposes. What can you hide when they can read your mind? This tech WILL fall into the wrong hands. An many many people will die from it. Think of place North Korea if this became cheap to mass produce.
  
  On the bright side, this could save a couple thousand. Comparison to the doomed millions. Wait, that's not a bright side at all.
• Cynic wrote on 2011-09-26 02:17
  I would either end up in a mental institute or some huge pity fest if this were ever to happen to my dreams, so I think I shall pass.
• PorcelainVienna wrote on 2011-09-26 05:03

  Quote from NomadTrooper;599925:

  I see this only being used for nefarious purposes. What can you hide when they can read your mind? This tech WILL fall into the wrong hands. An many many people will die from it. Think of place North Korea if this became cheap to mass produce.
  
  On the bright side, this could save a couple thousand. Comparison to the doomed millions. Wait, that's not a bright side at all.

  
  
  That is so depressing and frightening.
• Chiyuri wrote on 2011-09-26 10:23

  Quote from NomadTrooper;599925:

  I see this only being used for nefarious purposes. What can you hide when they can read your mind? This tech WILL fall into the wrong hands. An many many people will die from it. Think of place North Korea if this became cheap to mass produce.
  
  On the bright side, this could save a couple thousand. Comparison to the doomed millions. Wait, that's not a bright side at all.

  
  
  Well they can only read what you "see" inside the mind. With a bit of practice you can easily control what you "see" in your mind. Of course you will not be able to control it if you aren't concious but.. Dreams aren't exactly an easy book to decode..
• TropicalCat wrote on 2011-09-26 10:28
  Holy o.o
  
  Thats incredible.
• TA wrote on 2011-09-26 13:52
  Technically, they could also clear someone that was innocent of a crime by diving into their memories and finding what they were really doing that day.
  
  You guys are so pessimistic...
  
  I'm excited about the media applications and my own possible applications of such technology.
• Chiyuri wrote on 2011-09-26 20:24

  Quote from TA;600209:

  Technically, they could also clear someone that was innocent of a crime by diving into their memories and finding what they were really doing that day.
  
  You guys are so pessimistic...
  
  I'm excited about the media applications and my own possible applications of such technology.

  
  
  I dunno about you but, if I want to I can overload my memory with created imagery I decided on.. I could for example be guilty of murder and force my memory to only incorporate the vision of me trying to save the person who in reality I killed. People who see these memories would only think I was at the wrong place at the wrong time and I am not guilty. I could even image someone else and make them look like the killer, passing the blame to them.
  
  Memories are in the end under the control of who they belong to.
• Hiccup wrote on 2011-09-26 22:27
  I dun wanna see my dreams become reality ;__;
• Strawberry wrote on 2011-09-26 22:29
  I really don't want to see my dreams become reality, because they are pretty freaking ridiculous, lol. But then again, if someone had memory loss or whatsoever, what would end up happening there?
  
  And for some reason, someone with a gun always shoots me in my dreams most of the time. I don't even play gun games anymore, and I don't even. >>
• Juno wrote on 2011-09-26 22:41
  A century from now, people will be groaning about all the dreamspam on blogs.
• Tarvos wrote on 2011-09-26 22:49

  Quote from Juno;600712:

  A century from now, people will be groaning about all the dreamspam on blogs.

  
  
  What has science done!
  
  I think it's a pretty cool idea idea myself though. Anyone ever have a really cool dream, only to wake up and forget exactly what was going on, left with just a feeling of what happened? I'd love to see some of my forgotten dreams, especially some of the really bizarre ones.