I initially started fixing Babylon 5 at least 5 years ago. It proved to be a much bigger project than I anticipated and I didn't finish. However, due to recent interest in Babylon 5 from my friends, and some recent work I've done using code to process video automatically, I thought I might be able to take it up again, if I find there is any interest. I started this blog to at least let people know where the project stands.
Why does Babylon 5 need fixing?
At the time Babylon 5 was being produced, almost all television was being presented at a 4:3 aspect ratio (fullscreen). However, throughout the industry, and especially in science fiction circles, it was known that eventually there would be a change over to a 16:9 aspect ratio (i.e. widescreen, like modern TVs and most movies). Therefore all the live action scenes were filmed in 16:9 to be forwards-compatible. However, due to budget concerns, all the special effects were only rendered in 4:3. It was thought that when it came time to show Babylon 5 in widescreen, the original effects could be re-rendered for 16:9, perhaps even improved, as has been done with classic and next gen Star-Trek. Although all the relevant data was saved against such a day, the studio that technically owned all the property insisted that it be transferred to them for safe-keeping, which they failed at and it was all lost. Or at least they say it was lost. Personally I tend to believe they were not sufficiently invested in the project to actually look carefully enough when the time came to find it, especially given the cost of re-rendering all the CGI. Babylon 5 has a reputation for being something that its fans and creative team understood and loved, but that the studio didn't really "get" and that had a few prominently placed enemies who might have sabotaged it in just such a way.
However, Babylon 5 was nevertheless both shown and released in 16:9 widescreen aspect ratio. How was this accomplished? Any scenes that included special effects, whether they be CGI entirely or composited CGI over live action were taken from the existing 4:3 material produced for television and cropped at the top and the bottom to make them 16:9 widescreen. If you have watched the DVDs or the widescreen presentation of Babylon 5 on the SCI-FI channel then you have seen that this has the disconcerting effect of making some shots much tighter than they were originally intended to be, turning tight shots into close-ups, close-ups into extreme close-ups. You'll be watching along from a somewhat distant vantage point and suddenly be thrust much closer into the action, and then removed back again. Sometimes overlayed credits would even change size from one credit to the next.
During college, I was a member of a Babylon 5 fan-club. At the time, Babylon 5 was shown on a diverse array of differently affiliated channels all over the country, often preempted by sports, and it could be quite difficult to find and watch all of it. We maintained an archive of episodes from a variety of sources, which I still have access to. With such access, it seemed to me I should be able to remedy this, though it would be at the expense of some kind of intermittent frosted frame or black bars on the sides of the picture, that I'm sure the studio would never consider for release or showing, but would be what I personally would want to watch when re-viewing Babylon 5, so I set about 'fixing' it.
When I began, I thought this would be the only issue I would have to deal with and was prepared for it, but as it turns out there were many further issues that bogged down the project and ultimately led me to shelve it.
First, I wrongly assumed that the widescreen material had been filmed at the television standard of 30 frames per second. That is what I would have done, had I the option to do so. However, it turns out it was filmed at a cinematic 24 frames per second. Now if I HAD filmed it at 24 frames per second, I would also have produced the CGI at 24 frames per second, but it turns out the CGI was produced at 30 frames per second. Perhaps they believed that the new widescreen television standard would be 24 frames per second as well as widescreen and that when the CGI was re-rendered it could be done at 24 frames per second. I would have assumed (correctly) that the new television standards would support multiple frame-rates and that having the same frame-rate for all material would be better for the finished product in just about every way. But as I said above that never happened. Regardless, I was dealing with source material not only at two different aspect ratios but at two different frame-rates. Marrying it well was going to be a nigh impossible task. Why? Most 24 fps material is stored on media at 24 fps and converted if necessary to 30 for viewing on 30 fps only televisions by the DVD player, but left at 24 fps for displays that can handle it. But the 24 fps material on the Babylon 5 dvds is pre-converted to 30fps using a method that looks decent on old locked 30fps interlaced televisions, but suffers considerably when up-scaled by modern DVD and BluRay players and stored as such. I would have to choose between the CGI looking good on modern players or the live action looking good. The producers of the DVDs chose the CGI (probably because it needed all the help it could get given it was old and they were cropping it). But I wanted Babylon 5 to look better than that!
When I discovered this, I almost threw up my hands and gave up. But then I had a really weird idea that I decided to try. I decided to experiment to discover whether DVD players and DVD playing applications could handle streams that switched from 24 to 30 fps and back again. I knew that DVDs could contain information encoded at both framerates; menus were often stored at 30 fps while the movie on the same disc would be at 24, but I did not know if they could switch back and forth mid-stream without dieing. To my great surprise it turned out that all the ones I owned could! Assuming this was the case industry-wide, I could now store the information at whatever frame-rate was better for the source (on a scene-by-scene basis) and let the DVD player determine, given the framerate capabilities of whatever device it was outputting to, whether or how to convert it for presentation. This was amazingly ideal! As frame rate capabilities increased on televisions to match those already available on computer monitors, as the distinction between computer monitors and television began to blur anyway, my fixed Babylon 5 would always have been displayed at its best! And so, despite some misgivings about how complicated the process of producing DVDs that switched frame-rates had proved to be, I began the project.
However, I was simply not prepared for the sheer amount of CGI there was in Babylon 5. The first few episodes went quite well, but then they didn't have that much budget to begin with. As the show progressed and more funding and technology became available, more and more CGI shots of shorter and shorter length began to be used, some so short that the people who converted the show for DVD had missed them and they were being presented squished or stretched on the DVDs themselves rather than cropped. CGI began to be used to fix continuity and other issues with the live action material. Some CGI shots began to be produced at 24 FPS while others continued to be at 30 fps. Also the producers of the DVDs had not cropped all the CGI shots equally. In some, the top of the shot was used, in others the bottom. Sometimes they got fancy and panned from top to bottom or bottom to top over the course of the shot. Additionally, they had actually been re-edited for DVD. Very few shots
were actually removed or added, but the exact place of the cut would
often be a frame or two different between the DVD and the off-air tapes,
leading to sync issues. The color and dynamic range of the various source materials did not match and had to be futzed with. In short, it turned into a big old mess. Initially I was able to do an episode a day, then it became an episode every few days, and finally I was struggling to get an episode done in a week at which point I shelved the project having completed perhaps a dozen episodes.
However, at the time I was doing all of this by hand, looking at each shot individually and determining if it was completely live-action, completely CGI, or a mix of the two and what frame-rate it was at. I was working on a computer not capable of viewing any of this at full speed, or scanning through it quickly to make sure things synced up. Also now I have the ability to produce videos at higher custom frame-rates that people will be able to watch on modern computers and blue-ray players and need no longer rely on DVD players continuing to be able to switch frame-rates midstream without a hiccup. I also believe I have the capability to automate (or at least use algorithms to assist) this process to the extent I may be able to work at a decent rate of speed. Given all this, I may (and I say this with emphasis MAY) begin working on 'fixing' Babylon 5 again. If there is any interest.
How can you show your interest?
Well, currently, I spend much of my time authoring Minecraft videos for YouTube at http://www.youtube.com/user/ghohenzollern. Support of those videos or of the Babylon 5 project specifically would help me free up time to work on Babylon 5. As it is, I record and edit all the Minecraft videos myself (one per day), as well as contributing to the mod-pack we are using. This takes up all of my time, but a little support would allow me to purchase equipment that would speed both processes or perhaps hire someone to edit the Minecraft videos for me.
Also, there exist commercially-released VHS tapes of some episodes of Babylon 5. Our fan club did not have the resources to purchase more than a few of them, and most of those had been converted to PAL format for overseas broadcast and back again for import into the U.S, meaning to 25 fps and back to 30. As such they are not usable. If you have access to any commercially released NTSC format VHS tapes of Babylon 5 in 4:3, those could be of great value to me as our archive is missing bits here and there and is not of consistent or commercial quality.
Finally, should I manage to do all the work involved, there is the problem of how to distribute the results. I have no idea how to do so ethically or legally, I just think I'm in a position to finally solve the technical issues. If you are good at solving the other issues, I'd love to work with you to distribute the results.
Finally, I believe (though I am not 100 percent certain) that other shows (notably the first few years of Stargate SG-1) suffer from similar mixed framerate and aspect ratio problems and if there is interest in "fixing" those as well, I'd be glad to work on them also.
Where will I start? Rather than starting at the beginning as I did before, which made the project look far more possible than it turned out to be, I'm going to start in the middle, with a difficult and popular episode that's liable to generate some interest in the project and have all the problems an early episode could have as well as all the problems a later episode could, so I can try to solve as many of them as possible algorithmically and automatically from the very beginning. I figure if my code works with an episode like that, it should work with them all. And so, my intention, if I ever find the time, is to start with the 2-part episode from season 3, "War Without End" Parts 1 and 2. But that will only happen now if I get some interest. So let me know if you have any at ingresschmendrik@gmail.com.
