JMF wrapper for ffmpeg

 

http://fobs.sourceforge.net/

Try FOBS, the C++ & JMF wrapper for ffmpeg.

Omnividea Multimedia and the computer vision group at UPM-DISAM present FOBS: Ffmpeg OBjectS. FOBS is a set of object oriented APIs to deal with media. It relies in the ffmpeg library, but provides developers with a much simpler programming interface.

FOBS is currently available in C++. The Java side (Fobs4JMF) has been implemented as a JMF plugin that allows to use JMStudio as a Java Media Jukebox to play the most common formats and codecs (ogg, mp3, m4a, divx, xvid, h264, mov, avi, etc). Binaries for this enhanced version of JMStudio are available at the downloads section for Windows, Linux and Mac OSX. Developers can use the same binaries to include support for many more formats & codecs into their own JMF applications without changing a line of code!

 

A Mistake Asking to Be Re-Made

JMF: A Mistake Asking to Be Re-Made

Tuesday December 13, 2005 6:02PM
by Chris Adamson

AddThis Social Bookmark Button

Related link: http://archives.java.sun.com/cgi-bin/wa?A2=ind0511&L=jmf-interest&F=&S=&P=2852

Oh no, not again.

JMF, the Java Media Framework, has had a history that can honestly be described as alternating periods of ineptitude and neglect. The fact that the JMF home page has only seen three updates in the last two years, and none in the last twelve months, indicates that JMF is in the latter state.

And now this post to the JMF-INTEREST list:

I’m T— W—- and have recently taken responsibility for JMF at Sun
Microsystems.
We are in the process of planning what to do with JMF and would like
hear from you
regarding how you are using JMF and what your needs are. Please feel
free to contact
me directly as some of the other feedback channels are currently not
working properly.

Oh, where to begin?

OK, before I offer any more criticism, I need to acknowledge that I’m the author of a book on QuickTime for Java, a rival Java media framework. Some may think I’m trying to goose my own book sales. Think that if you like, but the book’s been out for a year and has probably sold most of the copies that it’s ever going to.

And let me add this: I started with JMF. If it were good, I might never have moved on to QTJ. After all, QTJ is limited by the fact that it only works on platforms with native implementations of QuickTime — meaning only Mac and Windows. An all-Java media framework would be tremendously valuable to the Java platform.

But I am absolutely convinced that Sun is in no way capable of creating such a thing.

The proof of this is in the results: since its release in 1998, JMF has gained practically no traction, and has been largely ignored since the release of JMF 2.0 in late 1999. We’ve gone six years with virtually no substantive work on the framework.

A little history as to how we got here… With no experience, credibility, or patents in the media field, one might have expected Sun to take on a partner in developing JMF, someone like Macromedia (Flash), Apple (QuickTime), or Real. Instead, they developed JMF 1.0 with Intel, and 2.0 with IBM.

JMF 1.0 was quickly pulled together to enable playback of dynamic media — audio and video — in Java desktop applications. JMF 2.0 added capture, streaming, and pluggability. But because of the high demands of media and the modest performance of late 90’s VM’s (and the capabilities of the hardware they ran on), all-Java media handling realistically needed to be bolstered by native “performance packs”, which improved JMF on supported platforms by using high-performance native code, and access to the platfrorm’s native media frameworks.

So… what’s the problem? Here are a few:

  • JMF has no editing API, nor any meaningful concept of media in a stopped state. That means it can’t be used for building, say, a podcast editor (can’t trim your clips), or iTunes (no metadata API for reading the track titles). Aside: what’s the point of a capture API if there’s no way to edit the captured data (apparently, the capture is only useful for streaming applications).
  • Sun made a performance pack for Solaris, that ubiquitous champion of the desktop, and not for Mac Classic or Mac OS X.
  • The included codecs supported few media types in common use at the time, and those that were used in 1999 (Cinepak) have fallen out of use.
  • The system for managing plug-ins, the “JMF Registry”, was extremely brittle.
  • The scheme for finding an appropriate plug-in used the wildly inefficient tactic of using exceptions for program flow.
  • Sun expected Macromedia to develop the Java support for the Flash format, and Macromedia lost interest, leaving JMF supporting only Flash 2.

Now it’s six years later and what’s been done? MP3 support was taken out of JMF for a few years due to licensing concerns, then put back in. Other than that, the framework has languished. Sun got interested in JMF again in July 2002, but they couldn’t actually hire anyone to work on it, and ended up not actually doing anything with it. So, developers come along, try to discover what it can do, and often wander off in disgust that it can’t work with modern formats. Some go to QTJ; many more probably call native frameworks with JNI, or just abandon Java altogether.

Now Sun wants to know what to do with it? Seriously?

Look, nobody developing a commercial application could risk Sun walking away from JMF for another six years. And given the utter lack of interest from third parties in extending this built-to-be-extended platform — it would be straightforward to bring Real to JMF via the open-source Helix platform, but nobody seems to have bothered — there’s no realistic chance of a third party coming to the rescue.

And seriously, what’s the point? Does Sun have a strategic vision for JMF? Is there some genuine value it can bring to the Java platform? Will putting dollars into new development pay off someday? Are these questions really going to be answered by casually throwing out a “Hi, what should we do with this?” to the handful of developers who are still hanging on?

For a lot of reasons, some technical, some not, JMF is a hopeless case. Unfortunately, due to the benefits of incumbency (the prominent javax.media package), it lures developers into a Venus Fly Trap of minimal functionality and unfixed bugs.

So, Sun, do you want to know what I think you should do with JMF? Deprecate it. Stop wasting our time. Tell everyone you’re done and move on to something that might work out better.

What good could come from reviving JMF?

http://adserver.adtechus.com/adserv/3.0/5159/463576/0/170/ADTECH;loc=300;key=key1+key2+key3+key4;grp=%5bgroup%5d

Comments

Hear hear!
I sure hope you emailed this to that guy, as Sun seriously needs to be told off on this. Sun already screwed the pooch by letting Flash become what it is today instead of Applets; JMF either needs to go away or be made current, and I believe what you said in doubting Sun cares enough to do it.

hexghost | December 13, 2005 06:43 PM

You missed the big problem completely
It was an interesting post Chris and while I agree with pretty much everything you said I think you missed the biggest problem that JMF and any “media” product faces: Copyright, trademarks and patents.
Its a mess out there, we recently did a project that required AMR support. Our team spent months with the patent holders for some key properties trying to get to a redistribution deal… Its just impossible to support all of these codecs and standards in a big business which is why only the “big” companies that have strong ties to the “industry” have products in this field.

So essentially the main problem is a legal rather than a technical problem and none of us (engineers) can solve it, or can we? We solved our problem by using QT4J but this eliminated Linux/Unix as a potential platform for us…

I raised a solution for this problem in my (old) blog a year or so ago http://jroller.com/page/vprise/?anchor=jmf_is_dead_jdic_should
in essense what I say in the entry is that we should rely on the fine work done by the teams at Apple, Real, Microsoft and MPlayer (or any other Linux/Unix player) by providing a reasonable API to interface into these tools. I think JDIC is the perfect vehicle since it allows Sun to maintain its “distance” from potential lawsuits that might follow.

Anyway we both definetly agree that JMF is dead and should stay dead, I do think that Sun needs to do something in this area though. Since most of us can’t rely on QT4J if only for the reason of its limited platform support.

vprise | December 14, 2005 06:22 AM

deprecate? No!
No, don’t do it! If JMF mean a wasting of time for Sun, makes what one becomes for all standard as this: It specifies it in the JCP and open it!

Copernico | December 14, 2005 06:44 AM

We need JMF or at least something similar
Hi, I am steering member of Javapolis http://www.javapolis.com. We have started last year to publish on the web videos of the Javapolis conference and will do the same this year.
When I developped the player, I was really willing to do it as a Java applet playing and navigating into MPeg4 content.
IBM had a good player, but not free, then there was an open-source player “MediaFrame” that was not stable enough at the time.
At the end, I have selected Macromedia Flash technology to develop the player. That was the first time in 5 years I was dropping Java for a new language.
I must say that I appreciate Flash for what it can do the best. Clearly, video and animations are really easy in Flash.
I do not think Java will ever compete with Flash, it is too late to jump in this bandwagon.
But definitely, multimedia capabilities are lacking in Java. We need at least MPEG-4, MPEG-2 and MPEG-1 video and MP3, AAC audio decoding with a decent API to be able to navigate into and get meta-data from that kind content.

mulkers | December 19, 2005 02:22 AM

dont stop
I just started with JMF and I am not unhappy with it. My customers are actually really happy with it.

Deprecation would make me wanna cry. I am currently using JMF in an applet. Pictures can be taken by means of capturing a frame. Installation and detection of devices goes remarkeable smooth. Also installing a VM goes well these days, the size of the VM is not a problem (dutch bandwidth). It is actually the first time for me that an applet adds something that could otherwise not be achieved. Maybe with flash but who wants to use that? I dont!

I am cool with java but the icing on the cake would be hot swap enhancement. A discussion about this on http://forum.java.sun.com/thread.jspa?threadID=572396&tstart=0

Anybody that agrees, pleas vote for it on http://bugs.sun.com/bugdatabase/view_bug.do?bug_id=4910812

Dennisb | December 29, 2005 12:53 PM

Try FOBS in conjunction with JMF
I have to say, I struggled with QTJava for a long time, just to get a simple player inside of a more complicated, resizable and proportional layout, to work properly. The movie would always disappear, or the control bar would disappear, or the program would crash, or it would play outside of its bounds, etc. The tricky part was that I needed to load multiple movies into this layout, which is when everything usually went awry. And then don’t get me started on QT updates which break everything, constantly. And installers! I do have access to the QTJ book 🙂

With JMF, it all worked the first time, without any hacks required, other than FOBS. FOBS is a JMF plugin which allows you to use FFMPEG, which supports all sorts of current formats.

http://fobs.sourceforge.net/

JMF also seems to me to be a very elegant, event-driven framework with good documentation. It can all be plunked into a single installer. Neither can be said about QTJ.

Plus this works on Linux, as well as Mac and Win.

So don’t get me wrong – I do all my work on a Mac, but JMF was much better for the seemingly simple tasks I needed it for. I know it’s not as powerful as QTJ when it comes to editing, but all I wanted was a reliable player!

Also, what was the problem with detecting a stopped state? I had no problems with that.

J

DiehardMM | January 9, 2006 02:42 PM

Hello, I just wrote a speeh to sms j2me programe using j2me.mmapi onf the handset and sphinx4 ( voice recognition ) and FFMpeg for Java bindings theses all support the JMF. Had not the JMF exisited could these two applications talk together? without glass in my food? no. JMF need to support things such as media signing, so say in my case the client licenses the AMR capture codec and then the license to use the AMR deocder on my server also. Why should I pay so they can decode their own content?

http://cmusphinx.sourceforge.net/sphinx4/
http://ffmpeg.sourceforge.net/index.php
http://www.mofokom.biz

Warren Crossing | February 22, 2006 07:55 PM

I just wanted to bring attention to our new effort to create an open-source implementation of JMF, (FMJ, fmj.sourceforge.net), which can hopefully address some of the problems mentioned. We are still heavy in the development stage, but are looking for help…

Ken Larson | June 2, 2006 05:44 PM

Java Media Framework

Java Media Framework

Introduction

The Java Media Framework (JMF) is a recent API for Java dealing with real-time multimedia presentation and effects processing. JMF handles time-based media, media which changes with respect to time. Examples of this are video from a television source, audio from a raw-audio format file and animations. The beta JMF 2.0 specification will be used for this report, as they currently reflect the features that will appear in the final version.

Stages

The JMF architecture is organized into three stages:

During the input stage, data is read from a source and passed in buffers to the processing stage. The input stage may consist of reading data from a local capture device (such as a webcam or TV capture card), a file on disk or stream from the network.

The processing stage consists of a number of codecs and effects designed to modify the data stream to one suitable for output. These codecs may perform functions such as compressing or decompressing the audio to a different format, adding a watermark of some kind, cleaning up noise or applying an effect to the stream (such as echo to the audio).

Once the processing stage has applied its transformations to the stream, it passes the information to the output stage. The output stage may take the stream and pass it to a file on disk, output it to the local video display or transmit it over the network.

For example, a JMF system may read input from a TV capture card from the local system capturing input from a VCR in the input stage. It may then pass it to the processing stage to add a watermark in the corner of each frame and finally broadcast it over the local Intranet in the output stage.

Component Architecture

JMF is built around a component architecture. The compenents are organized into a number of main categories:

  • Media handlers
  • Data sources
  • Codecs/Effects
  • Renderers
  • Mux/Demuxes

Media Handlers

MediaHandlers are registered for each type of file that JMF must be able to handle. To support new file formats, a new MediaHandler can be created.

Data Sources

A DataSource handler manages source streams from various inputs. These can be for network protocols, such as http or ftp, or for simple input from disk.

Codecs/Effects

Codecs and Effects are components that take an input stream, apply a transformation to it and output it. Codecs may have different input and output formats, while Effects are simple transformations of a single input format to an output stream of the same format.

Renderers

A renderer is similar to a Codec, but the final output is somewhere other than another stream. A VideoRenderer outputs the final data to the screen, but another kind of renderer could output to different hardware, such as a TV out card.

Mux/Demuxes

Multiplexers and Demultiplexers are used to combine multiple streams into a single stream or vice-versa, respectively. They are useful for creating and reading a package of audio and video for saving to disk as a single file, or transmitting over a network.

Presenting Data

The Java Media Framework provides a number of pre-built classes that handle the reading, processing and display of data. Using the Player, media can easily be incorporated into any graphical application (AWT or Swing). The Processor allows you to control the encoding or decoding process at a finer level than the Player, such as adding a custom codec or effect between the input and output stages.

Using the Player

The Player class is an easy way to embed multimedia in an application. It handles the setup of the file handler, video and audio decoders, and media renderers automatically. It is possibly to embed the Player in a Swing application, but care must be taken as it is a heavy-weight component (it won?t clip if another component is placed in front of it).

import java.applet.*;

import java.awt.*;

import java.net.*;

import javax.media.*;

public class PlayerApplet extends Applet {

Player player = null;

public void init() {

setLayout( new BorderLayout() );

String mediaFile = getParameter( “FILE” );

try {

URL mediaURL = new URL( getDocumentBase(), mediaFile );

player = Manager.createRealizedPlayer( mediaURL );

if (player.getVisualComponent() != null)

add(“Center”, player.getVisualComponent());

if (player.getControlPanelComponent() != null)

add(“South”, player.getControlPanelComponent());

}

catch (Exception e) {

System.err.println( “Got exception ” + e );

}

}

public void start() {

player.start();

}

public void stop() {

player.stop();

player.deallocate();

}

public void destroy() {

player.close();

}

}

In this case, we are using the static createRealizedPlayer() function of the Manager class to create the Player object. This ensures that the visual and control panel components are created before it gets added to the window by blocking until then. It is also possible to create an unrealized player and implement the ControllerEventHandler interface. The window then waits for the controllerUpdate event to fire and adds the components to the layout as they are realized:

public synchronized void controllerUpdate( ControllerEvent event ) {

if ( event instanceof RealizeCompleteEvent ) {

Component comp;

if ( (comp = player.getVisualComponent()) != null )

add ( “Center”, comp );

if ( (comp = player.getControlPanelComponent()) != null )

add ( “South”, comp );

validate();

}

}

Using a simple applet tag, a multimedia stream can easily be embedded in a webpage:

<APPLET CODE=PlayerApplet WIDTH=320 HEIGHT=300>

<PARAM NAME=FILE VALUE=”sparkle2.mpeg”>

</APPLET>

This will create an applet with an embedded MPEG video stream:

This can also be used to embed multimedia content in an HTML file, as shown below. Previously, browser-specific plugins were required.

Using the Player with Swing

The Player can be easily used in a Swing application as well. The following code creates a Swing-based TV capture program with the video output displayed in the entire window:

import javax.media.*;

import javax.swing.*;

import java.awt.*;

import java.net.*;

import java.awt.event.*;

import javax.swing.event.*;

public class JMFTest extends JFrame {

Player _player;

JMFTest() {

addWindowListener( new WindowAdapter() {

public void windowClosing( WindowEvent e ) {

_player.stop();

_player.deallocate();

_player.close();

System.exit( 0 );

}

});

setExtent( 0, 0, 320, 260 );

JPanel panel = (JPanel)getContentPane();

panel.setLayout( new BorderLayout() );

String mediaFile = “vfw://1”;

try {

MediaLocator mlr = new MediaLocator( mediaFile );

_player = Manager.createRealizedPlayer( mlr );

if (_player.getVisualComponent() != null)

panel.add(“Center”, _player.getVisualComponent());

if (_player.getControlPanelComponent() != null)

panel.add(“South”, _player.getControlPanelComponent());

}

catch (Exception e) {

System.err.println( “Got exception ” + e );

}

}

public static void main(String[] args) {

JMFTest jmfTest = new JMFTest();

jmfTest.show();

}

}

Capturing Real-time Data

Video and audio data can be captured in real-time from input sources and streamed to files on the local filesystem.

Capturing Audio

To capture audio, the specified sampling frequency, sample size and number of channels must be specified. JMF will attempt to locate any devices which will support this format and return a list of all that match.

CaptureDeviceInfo di = null;

Vector deviceList = CaptureDeviceManager.getDeviceList(

new AudioFormat( “linear”, 44100, 16, 2 ) );

if ( deviceList.size() > 0 )

di = (CaptureDeviceInfo)deviceList.firstElement();

Processor p = Manager.createRealizedProcessor(di.getLocator());

DataSource source = p.getDataOutput();

The source object returned from the Processor can then be turned into a Player object by calling Manager.createPlayer(). To capture it to an audio file instead, a DataSink can take the data instead:

DataSink sink;

MediaLocator dest = new MediaLocator(“file://output.wav”);

try {

sink = Manager.createDataSink(source, dest);

sink.open();

sink.start();

} catch (Exception e) { }

The combined source above will take input from the first matching audio device (usually a microphone) and stream it to a wave file on the local filesystem.

Capturing Video

Capturing video is identical to capturing audio. Most video sources have an accompanying audio track that must be encoded as well, so we must create a compound destination file.

Format formats[] = new Format[2];

formats[0] = new AudioFormat( “linear”, 44100, 16, 2 );

formats[1] = new VideoFormat( “cvid “); // Cinepak video compressor

Processor p;

try {

p = Manager.createRealizedProcessor( new ProcessorModel( formats, null ) );

} catch ( Exception e ) { }

DataSource source = p.getDataOutput();

MediaLocator dest = new MediaLocator( “file://output.mov” );

DataSink filewriter = null;

try {

filewriter = Manager.createDataSink( source, dest );

filewriter.open();

filewriter.start();

} catch ( Exception e ) { }

p.start();

This source will create a Quicktime-format file called “output.mov” with an audio track encoded raw and a video track encoded with the Cinepak compressor.

Conclusions

JMF is a highly flexible multimedia architecture that shows a lot of promise. In the future, hopefully Sun will work on making it more stable as well as on documenting the framework and providing more example code. The support for Video For Windows (VFW) makes it a good contender for future multimedia applications. In its current state, it is usable, but the lack of information makes it difficult to create a complex program.

Exercises

  1. Create a JMF-based teleconferencing system for two people to communicate over. Use RTP as the network communication protocol.
  2. Create a Swing-based TV watcher program. The program should allow the user to select the video device to use for capture.

Issues

  • Can JMF help propel Java into the field of multimedia display and editing?
  • Will Sun develop a Linux-native “performance pack” for JMF, even though Linux competes with Solaris?
  • What changes need to be made to the current beta version of JMF to make it more usable when it becomes a release?

References