SpaceX Restoration Video: A Digital Archaeological Dig

 

What’s Going On:

Last April, SpaceX attempted an ocean landing of the first stage, their first of a few since.  The flight was a resupply mission that was part of their contract with NASA to resupply the ISS with experiments and supplies as well as return experiments and garbage.   SpaceX has been flying several test flights of the Falcon 9R first stage using a first stage shell fitted with one Merlin engine.  From an initial test of just a six feet up to nearly 2500 feet, SpaceX has demonstrated the ability to translate and land the rocket on a predetermined point.  Next came the actual launch tests, fitted with landing gear, SpaceX attempted to “land”, essentially bring the stage to a controlled stop just above the ocean to demonstrate the ability to control the rocket all the way down.  Those tests didn’t include anything to land on, but that’s about to change with the next launch slated to attempt an actual landing on a barge out at sea, for a first ever recovery of a first stage rocket.  Pulling off such a move could dramatically reduce the cost of access to space because the recovery of a $60 million dollar component represents quite a savings.  Elon Musk likens it to air travel.  Who could afford to fly if airplanes had to be discarded after just one flight.

Bad Weather Ruins The Day…

Bad weather during the second attempt to land a stage at sea limited how closely NASA ships and planes could get to the action, hence, video transmitted during the landing was quite corrupted, not to mention, SpaceX could have done a few things prior to ensure better quality of the transmission of the video that they didn’t quite do.  It was a sad situation because, although SpaceX got the needed data to claim a victory for the experiment as a whole, the transmitted video was quite corrupted.  However, SpaceX did a few basic things to attempt to clean up the video, but the raw video itself, as well as SpaceX’s cursory attempt at video reconstruction didn’t yield much improvement.  In an attempt to solicit help from people outside of SpaceX, SpaceX decided to post both the raw and reconstructed versions of the video to the general public so that anyone, hopefully some “experts” in video reconstruction, could perhaps have better luck at repairing the video.

As Good As it Gets

Well, not exactly experts, but in steps a few forum members from Nasaspaceflight.com’s Web Form.  A post started with a discussion about the video, and weather or not anyone in the forum had any idea where to start.  Slowly, as people started reading the thread, a poster here and a poster there started to chime in, and fairly quickly, a large number of users were buzzing about what could be done to recover, is possible, any of the video of the ocean landing.  Before long, some headway began to be made, and a few folks with some knowledge, and a few with none, but quickly learning (me, and others as well), were exchanging information and sharing what we had figured out.  What this rag tag collection of amateur video buffs needed was a digital “Indiana Jones” to help all these amateur digital archaeologists in their quest for the holy grail, a view-able version of the badly damaged landing video.

Enter: Indiana Jones

One of the forum members realized that one of the other forum members was none other than Michael Neidermayer, the current curator of the widely popular FFMPEG video coded.  FFMPEG is an open source coder-decoder for MPEG video, and having the current curator onboard was a huge bonus.  Some of the more technically oriented members started asking Michael if he could add some command line switches to enable a user to manually control the position of macroblocks and manually set the chromatic values, and Micheal would run off and code that into FFMPEG and release the newer version with functions never before really needed, or asked for.  This turned out to be just what we needed, because in all honesty, the vast majority of the data received was good, but the nature of MPEG means, while it’s highly compressed data, which is good for video transmission, it’s bad for handling data errors, even smaller ones.

Back To School

Without going into too much detail, MPEG compression uses a truckload of compression tricks to reduce the amount of data needed to transmit video.  A standard film strip incorporates no compression at all.  24 to 30 frames per second (or more) of photographic images are strung together to make a viewable video.  For MPEG, that’s just not enough.  First off, full frames (“I-Frames”), contain all of the information needed to construct a frame, just like a JPEG still taken on your cell phone or digital camera.  Besides that are two types of predictive frames “P-Frames” and “B-Frames”, which predict motion using the previous frames (P-Frames) or both previous and future frames (B-Frames), or bi-directional frames along with the previous I frame to construct the in-between the I Frame frames.  P and B frames which carry info about objects that have moved, so that instead of re-sending data of the same object in a different spot, P and B frames say “use this part from the I frame, but move it over here”.  All these tricks serve to highly compress data to make digital video managable over bandwidth limits that we have to work with these days, along with flash and hard drive storage considerations as well.  The one drawback is that it makes video highly susceptible to even single bit errors.  It’s alot like a house of cards.  You pull one card out and it can bring down half or even all of the cards into a pile of cards that when viewed just looks like garbage.  The thing is, with a house of cards, if you put the card back, it doesn’t restore the house of cards, but with MPEG video… it does…  So, by manually moving around macroblocks, if we got lucky, because, well, it’s just a matter of picking a spot, and moving a macroblock to a location and seeing if the results improve the image or not, then we’d get an improved image.  The vast majority of the work done was to reconstruct these informational “I” frames, so, that when we were done, we could re-encode the video and let FFMPEG re-create the P and B frames from the restored “I” frames.

ALL ABOARD!

Forum members started to contribute more than just time to the project.  Some members started to produce web tools to aid in the reconstruction process.  To be a part of the fun, one had to download the file that contained the extracted damaged I frames, download the new mpeg hacker versions of FFMpeg put out by Micheal Neidermayer, and there was alot of manual work involved.  One forum member put together a web page that automated this process by allowing a user to simply go to a website and start punching in FFMpeg command line switches and data values to move around macroblocks, and the page would do the rest and instantly update the image so you could immediately see the result cutting out many steps, and opening the door to even more people jumping in, as anyone, forum member or not was welcome to try and contribute to the process.  By this time, SpaceX had taken notice of what we were up to, and an official SpaceX contact joined the forum to assist in any way SpaceX could, and pass along the  well wishes from everyone at SpaceX in our efforts.

Putting It All Together

After a couple of months of a small band of about 40 to 50 heavy contributors as well as an unknown number of additional work done by others (I’m not entirely sure of the total number of contributors though), a full blown reconstruction video was released to SpaceX as well as posted on YouTube.  It was really cool so see a rag tag bunch of interested forum members come together to collaborate on a pretty complex and work intensive reconstruction project, but it’s nice to see that happen and come together right before your very eyes.  By the time it was over, the thread had dozens of pages and well over 100,000 views.  SpaceX was really grateful for our work, and have used the reconstruction video as well as sent out kudos to everyone involved.  For me, being involved so closely in the project was alot of work, but it was also alot of fun, and it was really cool to be involved with the many many others, many of which contributed far more than myself (Personally, my biggest contributions was getting lucky with 2 of the 15 I frames and contributing some short command line codes that recovered significant portions of a couple of the frames, but I honestly thought I could have contributed more, and was surprised (and happy) to see my name listed as a contributor in the final restoration video.  I’m a big fan of SpaceX and it’s Iron Man leader, Elon Musk, so I viewed it as a real honor just to be involved in the whole project, which in many ways is better than any sort of monetary benefit I could have received.

SpaceX Landing Restoration Video

Good Luck SpaceX

SpaceX’s history is littered with naysayers all along the way that would swear they couldn’t get into orbit, and then after they did that, swear they would never recover a craft, until they did that, and then swear they’d never service the ISS, until they did that.  Do you see the pattern?  Yep, there’s folks out there swearing SpaceX will never recover the first stage.. which is great, because if thing keep going the way they are, they will do it.  Maybe not on the next launch, but the manifest is pretty full for 2015, and odd are one of the attempts this year will bring home that sausage.  My money’s on SpaceX actually, because despite the naysayers, they tend to get where they’re going.

Advertisements