THE BOX MOVE
  • Home
  • Blog
  • Profiles
    • Nimra Asi
    • Hassan Bukhari
    • Kamil Ahsan
    • Mehr Un Nisa
    • Maheen Rashid
    • Nayab Abidi
    • Anum Khan
    • Mehdi Mujtaba
    • Haris Baig
    • Syed Ali Raza
    • Abdullah Khalid
  • Essays
    • Biology
    • Physics
    • Mathematics
  • Publication
  • Contact

Unflattening Pictures

11/6/2011

8 Comments

 
Picture
GCU
Note: This blog post is about Maheen's Senior Project. Since she graduated a year before the rest of the us (who are '12 students), her Project was completed earlier this year.

When a camera takes a picture, it’s “flattening” the world it’s viewing. So while it might be pretty obvious to us looking at this picture that the building is upright, and the ground is not, it is not that obvious to a computer. It does not know what buildings are, what trees are, what the ground is, what the sky is, and how all of these usually relate to one another. To your computer, this picture on the left is a two dimensional matrix. Without knowing anything about the three dimensional scene this picture captured, it would not be possible for your computer to infer anything about the nature of that scene. That information has been lost in the flattening.

Now, given certain constraints on the scene that has been flattened, it’s possible to recover the information your picture “flattened” out. Here are some examples of “unflattening” I really like.

There is research being done on this problem of “unflattening” a picture– more commonly known as single view 2D to 3D reconstruction - in the LUMS Computer Vision Lab. The constraints placed on the scene to be rectified were two. First, that the scene comprised of planes that were not discontinuous – so natural objects such as animals, mountains, plants, etc, are out ruled. Secondly, that each of these planes had some orthogonal lines on it, for example, a house with windows on each side. Using these two constraints (and some user input), single view reconstruction is possible.

Our senior year project was about doing the same thing, 3D reconstruction, but using information obtained from multiple images as opposed to a single image. By using multiple images we could create reconstructions that were more complete and real to life, since we can know what the object looks like from multiple angles and not just one. At the same time, we would also need to know how these photographs relate to one another so that we could join their respective three dimensional reconstructions together; there would have to be some kind of overlap between the images we use.

Now, we’ve assumed that our individual single view reconstructions work. Hence, we know that the constraints under which single view reconstruction is successful hold: we know that the scene we’re reconstructing comprises of planes that are not discontinuous, and that each of these planes has some orthogonal lines on them. By knowing this, we know that bringing two models together is just a matter of fusing together the two planes that are common between these two models. By common I mean both these planes are separate reconstructions of the same plane in the real world (that’s the one we live in I think).

It turns out that the minimum number of point correspondences you need to fuse together two planes is three. And that’s how much overlap you need in your photographs: three points. That’s it. 

Picture
Picture
Of course, once you’ve fused together your planes, you will not get a reconstruction that’s perfect. Your single view reconstructions (SVR) were not perfect to begin with, so your multi-view reconstruction (MVR) will also be imperfect, and it will only get more imperfect with every additional SVR you use.

Oh dear.

Picture
AnimalPictures1.com
Conversely, the more SVRs we use, the more information we have about the structure we’re reconstructing. So, shouldn’t our MVR get better, and not worse? Well, yes. But only if we use this extra information.

 We used the three (or more) common points to make all six of the points lie on the same plane. However, just because these points now lie on the same plane does not mean that these points themselves have fused together. We had six points on two separate planes to begin with. We will now have six points on the same plane – where as ideally, since these points represent the same three points in the real world, we should have three. Therefore, we try to construct our SVRs in such a way that the distance between corresponding points after stitching is minimized, that our six points come as close as possible to becoming three points.

However, if we remodel our SVRs to minimize common points’ distance only, we are also running the risk of creating worse models. Why? Because we are no longer taking in to account the nature of the scene we were reconstructing in the first place. In other words, we are not making sure that each plane has some orthogonal lines, and that planes are continuous and intersect each other properly. It is the interplay of all three of these factors that would create an accurate and realistic multi view reconstruction of the scene we plan to reconstruct.

Turns out this interplay is not that easy to figure out. Determining which factors need to be given relatively more weight depends on the quality of our SVRs, which in turn depends on the pictures and what they are capturing – and that’s the problem in the first place: we don’t know what exactly we’re taking pictures of. So we need to change our solution so that the question of weights does not come up, and that is what I am working on right now. However, by the end of our senior year, we hadn’t discovered this problem. We were playing around with weights and generating models that made us happy to look at. Here’s an example. The first two images are of what we were trying to stitch together. We were being lazy, so we just stitched multiple SVRs from this picture rather than multiple SVRs from different pictures. The second two are what the results of stitching looked like when we stitched the models together without optimizing. And the last is what we got after we took common points’ distance in to account and tried to minimize it.

Picture
Picture
Picture
Picture
Whenever I tell people about my senior project, they ask me of what practical use it is. Frankly, I don’t know. I’m sure somewhere someone will find this work useful, and not just to make an iPhone or Android application. Maybe historians would use it, or architects would. What difference does it make? It’s an interesting problem, and we tried to solve it. If it has any practical value, that’s a bonus. Who knows what practical value anything will eventually have? Now here’s the video I wrote this paragraph for: 
Maheen
8 Comments
    The Box Move blog is no longer active since the founding team has graduated. The archives will remain online.


    RSS Feed


    AFFILIATES:
    Picture
    The SPROJ Forum for the SSE 2012 batch. Discuss potential Senior Projects here.

    Picture
    Brain Talk is an online resource and forum for all things Psychology and Neuroscience.


    Picture

    MUSIC FOR GEEKS:
    Featured: Art Tatum Three Letter Word

    Like us on Facebook!


    ON THE FRINGE:

    Picture
    Learn something
    The story of a how a YouTube video of a blind man biking down a mountain inspired good non-fiction writing on echolocation. You may find it useful for your own writing. Read here.

    Picture
    An awful waste of space?
    Amidst NASA's budgetary cuts and scientists' renewed vigor in justifying Space Programs, it is important to shed some light on the background. Click here for a succinct overview of the Search for Extra-Terrestrial Intelligence(SETI) project.
    Picture
    Manto ka Muqaddama
    Pakistaniat.com publishes, on the anniversary of Saadat Hasan Manto's death, a sampling of his works, a tribute to him as well as articles chronicling the obscenity trial he was tried for. Read all three parts of the series here.
    Picture
    Not Another 2010 List
    The folks over at The Last Word put up their list for the best non-fiction in the past year, including 'The Mind's Eye' which is very hard to find in bookstores, indeed! Read the full list here.
    Picture
    Bird Conspiracies
    Leslie Kaufman at The New York Times tells us exactly why those birds flying above are dropping to their deaths. Click here to read the article.
    Picture
    Spontaneous Solar Growth!
    Reported at MadScience, scientists at MIT have found a way to create solar cells that can regenerate themselves like living organisms. Read more here. 
    Picture
    Lessons from Chernobyl
    Decades after the radiation disaster at Chernobyl, scientists elucidate how plant life has been thriving in the highly radioactive environment. Read more here.
    Picture
    Rolling Ribbons
    MIT Scientists revisit Galileo's famous inclined plane experiment, this time with polymer ribbons and discover complex results. Read here.
    Picture
    A Lifetime, Washed Away
    Pakistani author Daniyal Mueenuddin writes in the NY Times about the aftermath of the flood and displaced people. Read more here of the article posted by 3QD.
    Picture
    On String theory and Materials Science
    Click here to find out how physicists at MIT are using ideas of gauge/gravity duality to explain properties of superconductors.
    Picture
    That's why you're irrational!
    Newsweek's Sharon Begley provides a fascinating argument for why evolution may favor irrationality. I particularly liked the examples she picked. Read
    here.
    Picture
    Just when you wanted a gene kit
    The US Food and Drug Administration held hearings n the 19th and 20th of July to talk about the validity of tests which were sold directly to the public which gives consumers direct access to to their genomes. Should it be regulated? Read more here.
    Picture
    On Trees and Prisons
    In a 6 minute talk on Ted.com, Nalini Nadkarni (shown above) talks about her ideas of incorporating conservation into prison programs. Watch the talk and read Nadkarni's fascinating biography here.
    Picture
    These Lungs are made in USA
    Stem Cell Biology takes huge leaps forward with the new advances made in lung transplants based on using the lungs extracellular architecture. Read more from Nature here.
    Picture
    Economist Special Reports
    Ten years after Craig Venter revealed the first working draft of the entire Human Genome, this special report demonstrates how Biology is now at the brink of something brilliant - just recently, the draft of the entire genome of the Neanderthal was revealed. Suck on that, sceptics!
    Picture
    Orbit Stories
    Bobby Satcher, astronaut, the first orthopedic surgeon in space. Read all about his tales here on MITnews.
    Picture
    Craig Venter Creations
    Researchers create world's first fully synthetic self replicating, living cell. Massive fuss about limitless monster potential possible. Read the NewScientist article here. Watch the Ted.com talk by Craig Venter here.



    TAGS

    All
    3d
    500 Days Of Summer
    Abdus Salam
    Ageing
    Alan Turing
    Albert Einstein
    Aldous Huxley
    Algorithms
    Altruism
    A Mathematician\'s Lament
    Andrew Marvell
    Anfinsen\'s Experiment
    Angelina Jolie
    Angiogenesis
    Artificial Intelligence
    Atomic Bomb
    Autosomal Dominant
    Ayn Rand
    Baluchistan
    Barack Obama
    Bats
    Battlestar Galactica
    Beta Pictoris B
    Billie Holliday
    Biotechnology
    Bioterrorism
    Birthday
    Black Holes
    Blasphemy
    Blue-Footed Boobies
    Brian Greene
    Cancer
    Carl Sagan
    Cern
    Chaos Theory
    Chemistry
    Citizen Science
    Citrus Greening
    Climate Change
    Clinical Trials
    Cloud Computing
    Computer Viruses
    Computer Vision
    Comsats
    Copenhagen Summit
    Coping
    Creamed Spinach
    Creationism
    Cs101
    Cults
    Dark Matter
    Darwin
    Debating
    Deduction
    Delauney Graphs
    Democracy
    Depth Sensing
    Die Grunen
    Discrete Geometry
    Dna
    Dna Barcodes
    Dr. Abu Bakr Muhammad
    Dr. Asad Naqvi
    Dr. Sabieh Anwar
    Dr. Shahid Khan
    Dublin
    Edge Detection
    Education
    Education In Pakistan
    Ella Fitzgerald
    Emile Zola
    Energy Crisis
    Engels
    Enhancers
    Entropy
    Enzyme Kinetics
    Epigenetics
    Ernst Haeckel
    Espresso Taster
    Evo-devo
    Evolution
    Facebook
    Fairytale
    Faiz
    Feynman
    First-order Differential Equations
    Flood
    Fm Phone Transmitter
    Fourier Transforms
    Fractals
    Francis Bacon
    Freshmen
    Freud
    Fruit Flies
    Gadgets
    Gannets
    Gender
    Gene Clustering
    Genetic Engineering
    Genetics
    Google
    Graduate School
    Graph Generators
    Guinea Worm Disease
    Hadron Collider
    History
    Human Epigenome Project
    Humanities
    Hyposmia
    Image Processing
    Induction
    Inherit The Wind
    Insulin
    Invisibility Cloak
    Ipcc
    Isabel Allende
    Jacob Bronowski
    James Watson
    Jorge Luis Borges
    Journalism
    Kelly Clarkson
    Kinect
    Kings Of Convenience
    K-means
    Lisa Randall
    Lois Mcmaster
    Longevity
    Lorentz System
    Malaria
    Manhattan Project
    Marfan\'s
    Markov Chains
    Marx
    Mathematical Biology
    Mathematical Proofs
    Mathematics
    Matlab
    Media
    Mendel
    Meta-Consciousness
    Meta-flex
    Michio Kaku
    Miles Davis
    Mitochondria
    Mmr Vaccine Controversy
    Model United Nations
    Motion Detection
    Mozart
    Mri
    Multidrug Resistance
    Muons
    Music
    Musings
    Muslim World
    Nasa
    Nathiagali
    Nazca Boobies
    Neurodegenerative Diseases
    Neuroscience
    Noise
    Nuclear Fission
    One Laptop Per Child
    Optical Dispersion
    Owl City
    Paul Lockhart
    Penelope Cruz
    Penrose Stairs
    Pervaiz Hoodbhoy
    Photography
    Photolyase
    Photosynthesis
    Phrenology
    Physics
    Physics Envy
    Pineapple Chicken
    Pink Floyd
    Planet Hunting
    Pleasure Science
    Poetry
    Population Demographic Collection
    Positive Feedback
    Prions
    Processors
    Prolog
    Psi Fi 2010
    Psi-Fi 2010
    Psychology
    Psyllids
    Publication
    Quantum Physics
    Quarks
    Ramachandran Plots
    Recipes
    Recursion
    Redundancy
    Regulons
    Relief Efforts
    Religious Minorities
    Richard Dawkins
    Richard Feynman
    Rna
    Rna Interference
    Robert Frost
    Robotics
    Roger Penrose
    Role Playing
    Romanticism
    Romeo And Juliet
    Rudyard Kipling
    Rugby
    Scientific Journals
    Scopes Trial
    Self Assembly
    Self-assembly
    Semester Finals
    Senior Projects
    Sentiment Analysis
    Sherlock Holmes
    Signals And Systems
    Single View Reconstructions
    Sir Edward Dyer
    Slaughterhouse-Five
    Snow Leopard
    Sokal Affair
    Sse
    Stalin
    Stephen Hall
    Stephen Hawking
    Stephen Jay Gould
    Stephen Pinker
    Stress
    Superpowers
    Symmetry
    Systems Biology
    Taipei
    Taste
    Ted
    Telomeres
    The Butterfly Effect
    The Doors
    T.H. Huxley
    Third Eye Blind
    Thomas Nigel
    Time Traveling
    Tony Judt
    Topic Analysis
    Tracking Devices
    Transmitter Circuits
    Transposable Elements
    Turing Test
    Twilight
    Twistor Theory
    Ulysses
    Valentine\'s Day
    Vocational Training
    William Paley
    Women
    World Cup
    World Of Warcraft
    Wwf
    Zaid Hamid
    Zardari
    Zoo


    FEEDS:

    ARCHIVES

    January 2013
    August 2012
    June 2012
    March 2012
    February 2012
    December 2011
    November 2011
    October 2011
    September 2011
    August 2011
    July 2011
    June 2011
    May 2011
    April 2011
    March 2011
    February 2011
    January 2011
    December 2010
    November 2010
    October 2010
    September 2010
    August 2010
    July 2010
    June 2010
    May 2010
    April 2010
    March 2010
    February 2010
    January 2010
    December 2009

Powered by Create your own unique website with customizable templates.