A manifesto on the future of image coding - JPEG Pleno
Tremendous progress has been achieved in the way consumers and professionals capture, store, distribute, display and ultimately use images. We have been witnessing an ever growing acceleration in creation and usage of images in all sectors, applications, products and services. This widespread and still growing use of images has brought new challenges for which solutions should be found. Among others, image annotation, search and management, imaging security and in particular privacy, efficient image storage, seamless image communication, new imaging modalities, and enhanced imaging experiences are just a few examples of challenges to which scientific community, industry, service providers, and entrepreneurs have responded in the past, including continuing improvement of existing solutions and creating new ones.
During 25 years, Joint Picture Experts Group (JPEG) has been an example of such efforts, and has offered image coding standards which can cope with some of the above challenges. This has resulted in a series of successful and widely adopted coding algorithms and file formats such as JPEG and JPEG 2000.
JPEG format is today a synonymous of modern digital imaging, and one of the most popular and widely used standards in the history. Images created in JPEG format now exceeds 1 billion per day in their number, and most of us can count a couple, if not more JPEG engines in devices we regularly use in our daily lives; in our mobile phones, in our computers, in our tablets, and of course in our cameras. JPEG ecosystem is strong and continues an exponential growth for the foreseeable future. A significant number of small and large companies created in the last two decades have been relying on JPEG format, and this trend will likely continue.
A question to ask our selves is: will we continue to have the same relationship to flat snapshots in time (the so-called Kodak moments) we call pictures, or could there be a different and enhanced experience created when capturing and using images, that could go beyond the experience images have been providing us for the last 120 years?
Several researchers, artists, professionals, and entrepreneurs have been asking this same question and attempting to find answers with more or less success. Stereoscopic and multi-view photography, panoramic and 360-degree imaging, image fusion, points cloud plus texture imaging, high dynamic range imaging, integral imaging, light field imaging, and holographic imaging are among exemples of solutions that have been proposed as future of imaging.
The offered experience based on the above solutions obviously depends largely on the state of the maturity of technologies behind them, and their implementation, including the content created by them, the type of usage and relationship they offer to professionals and consumers. It is normal to expect that with enough efforts and resources, as well as enough time, some will have the potential of reaching a level of maturity that can make them good candidates to offer an experience beyond what exists today.
However, the way such solutions are introduced has probably as much importance as the solutions themselves. The JPEG ecosystem is huge, and offers sufficient experience for what it is expected to offer. The legacy that JPEG brings should not be underestimated. In fact, several attempts have been made in the past by companies, as well as by standardisation organisations, including the JPEG committee itself, to replace the JPEG format by an alternative. Success has been limited if not inexistent in many of such attempts.
Introduction of new imaging experiences should therefore be done in a smooth and gradual way without disruption of the existing JPEG ecosystem, but by enhancing it for those wanting it, while still offering the old experience to those who don't. JPEG backward compatible standards are therefore essential in introduction of new imaging standards, and should be given the attention they deserve, when assessing future image coding standards.
Recently, JPEG standardisation committee has created an initiative called JPEG Pleno. “Pleno” is a reference to “Plenoptic” a mathematical representation, which not only provides color information of a specific point in a scene, but also how it changes when observed from different directions and distances. “Pleno” is also the latin word for “complete”, a reference to the vision of the JPEG committee that believes future imaging will provide a more complete description of scenes well beyond what is possible today.
The road-map for JPEG Pleno follows a path that starts in 2015 and will continue beyond 2020, with the objective of making the same type of impact that the original JPEG format has had on today's digital imaging starting from 20 years ago. Several milestones are in work to approach the ultimate image representation in well-thought, precise, and useful steps. Each step could potentially offer an enhanced experience when compared to the previous, immediately ready to be used in applications, with backward compatibility. Backward compatibility could be either at the coding or at the file format level, allowing an old JPEG decoder of 20 years ago to still be able to decode an image, even if that image won’t take full advantage of the intended experience, which will be only offered with a JPEG Pleno decoder. Examples of potential milestones include panoramic and 360-degree images, fused images, and light-field images. Stay tuned as JPEG committee clarifies and shares its plans and achievements for JPEG Pleno in the coming months and if you want to join this effort, do not hesitate to drop me an email to Touradj.Ebrahimi@epfl.ch.
A manifesto on the future of image coding - JPEG Pleno
Privacy is one of those things intimately linked to individualism, and one of the fundamental needs of any human being. Obviously, the definition of privacy and its degree, or even its importance, largely depends on ones personality, age, and culture, among others. Recently, with the proliferation of video surveillance systems across cities, issues such solutions created for the privacy of individuals have been raised, and are among hot topics in discussions and debates around video surveillance. Other new information and communication technologies are also bringing their share of invasion of privacy. Recently, The Swiss Federal Commissioner for Data Protection announced they are taking Google to court because of the invasion of privacy that Street View represents. Likewise, serious privacy issues have been identified in Facebook, Youtube, Flicker, and many other social networking services. In fact, it seems that some of today’s services and products, and many more in future, in order to be good and efficient, will need to invade our privacy! Or this is at least what some pretend. Because of the undeniable added value of many of such products and services, some advocate that we should revisit our perception about privacy and live with the fact that it will be increasingly difficult to maintain privacy as we used to. In other words, we will have to trade our privacy for more features and functionalities, and often, we won’t be given even the choice. Interestingly, a non-negligible portion of scientific community and opinion leaders seem to agree! Well...I don’t. After scarifying QUALITY to get cheap goods, here comes loosing our PRIVACY to get goods. Living in a country which is build around the concepts of quality and privacy, I beg to differ! First of all, technology can be used to reduce, if not to eliminate the dilemma of making a choice between privacy and additional features. It is possible to have the best of the both, if an effort is made to do it. Second, and more importantly, this road will take us to a society where individuals will increasingly disappear in the collective. Trading privacy for services will lead into sharing all thoughts, and inevitably, some other individual values such as ownership. Do you know what a society deprived of privacy, ownership and other individual values is called: THE BORG COLLECTIVE!
Are you ready for assimilation?
I used to be quite a regular blogger for more than two years. I started my first blog on October 20th 2007 (see https://blogs.epfl.ch/touradj.ebrahimi/2007-10-20), and shortly after, on February 2nd, 2008, mirrored it on http://touradjebrahimi.blogspot.com/. Since joining facebook, which takes a good portion of the time I had put aside for my presence on blogosphere, the frequency of my blogs have reduced from once a week to once every 4 months! In fact, my last blog dates back to July 25th, 2009. I have decided to again give it a try and attempt to maintain a blog. This time, I am using Apple’s iWeb and MobileMe which seems to have an efficient user interface, hoping this would save me time, and will allow me to do both my blogging and facebooking in the same length of time I have devoted to such activities. A novelty for this new blog will be that as opposed to my past two blogs (which will continue to mirror this one for a while), I won’t only discuss my work related issues, but also extend to a much wider range of issues. Let’s see how it will go ...
This is a mirrored version of my new blog on http://www.ebrahimi.ch
It has been quite some time since my last blog. There are many reasons behind this. One is social networks such as Facebook, LinkedIn, and Twitter. I increasingly use these tools to communicate with my colleagues, friends, and all persons with whom I share similar interests. Of course these social networks do not completely replace the need for a blog, but definitely they answer to some of the needs which make people write blogs. And they have features that a blog cannot offer. Obviously, like many others the time I can devote to such activities is limited, and doing one thing more will necessarily leaves less time for something else. But I have not completely given up on blogging and you will see more blogs coming from me, at a slower frequency, but for sure on regular basis.
One other reason which has prevented me to write more blogs is QoMEX 2009. This is a workshop on Quality of Multimedia Experience that we founded about a year ago. The preparation of the workshop took a lot of hard work for everybody. I am glad to say that all is now ready for this event to take place next week! We already have nearly 100 confirmed participants, and an exciting program has been put in place. I will certainly write at least one more blog about QoMEX 2009 after it has happened. However, this may take some time. If like me, you are also a member of Fracebook, LinkedIn, and Twitter, go to QoMEX 2009 website at http://www.qomex.org and from there find the links to QoMEX at your favorite social network. I strongly recommend that you register to more than only one of these links as each bring a complimentary information. For example, Facebook QoMEX contains pictures, video, information, and opinion of participants. Twitter will be used more during the event to allow people to give on the fly their impressions, to send news around about events, papers, sessions, etc., and to share some real-time pictures, and more. LinkedIn is more for discussions before and after the event. Of course somethings can be done in all three. In additon to these, we have also created a presence for QoMEX 2009 on YouTube and DailyMotion with some interesting video already uploaded there. We have planned to record various sessions and presentations at QoMEX 2009 for those speakers who will allow us to do it, and defnitely some of those will find its way on these forums. We are working on a solution to put all the recordings on a site to allow those who could not attend QoMEX 2009 to access them freely, together with the slides of the presentations.
One of the secrets behind the success of social networks is that they live only if the community they are created for becomes active in them. So, please do not hesitate to becomes active in all the above social networks if you have an interest in multimedia and quality of experience. You can use QoMEX presence on these social networks to present yourself and your interests and activities, talk about problems you may have encountered, seek answers for them or present the solutions you may have found that resolve these problems, upload graphics, pictures or video of you or your work, and in short become a member of the community of experts in multimedia and quality of experience.
To end this blog, let me share with you some highlights of QoMEX 2009:
QoMEX’09 features oral presentations, exhibits, panels and poster sessions in order to provide attendees with various channels to exchange and acquire information about the latest developments and future trends in the field of multimedia user experience.
Highlights from the Technical Program:
“Innovating the Multimedia Experience”
Speaker: Mr. Dave Blakely, Senior Director, IDEO
“Haptic Design Guidelines and Tools for the Next Generation of User Experience”
Speaker: Dr. Christophe Ramstein, Chief Technology Officer, Immersion Corporation
“Light is a Two-Way Street: The Next 50 Years of Video”
Speaker: Dr. Bruce Flinchbaugh, Texas Instruments Fellow and Director of the Video & Image Processing Laboratory, TI, Dallas
“Comparison of Subjective Assessment Protocols for Digital Cinema Applications”
Speaker: Prof. Christine Fernandez-Maloigne, Professor of Signal and Image Processing in Poitiers University, France
More details about the talks:
Panel on “Quality of Experience: Tools, Targets and Trends”
Panel Chair: Prof. Fernando Pereira, Prof. ECE, Instituto Superior Técnico, Portugal
Prof. Alan Bovik, Prof. and Chair, ECE, UT Austin
Dr. Gary Sulivan, Video Architect, Microsoft and Chairman, ITU/VCEG
Prof. Sebastian Moeller, Deutsche Telekom Laboratories and Berlin University of Technology
Dr. Stephen Winkler, Principal Technologist, Symmetricom
More details about the panel
Technical papers will address the following major areas:
• User Experience Assessment and Enhancement
• Visual User Experience (Image/Video/Graphics)
• Auditory User Experience (Speech/Audio)
• Standardization Activities in Multimedia Quality Evaluation
See here for the list of papers
To register for the workshop please use this link
A PDF version of the program guide and further information is available at:
DSS 2009 took place between 13 and 17 April 2009 in Orlando, FL. More information about the event and its various conferences are available at:
I had two talks at DSS 2009 this year, both at the Mobile Multimedia/Image Processing, Security, and Applications conference. My first talk was on Next Generation Image Coding Standards: JPEG XR and AIC, was an invited talk. The second was entitled: Multi-view Video Segmentation and Tracking for Video Surveillance.
You can consult the slides of the invited talk below. As the link is sometimes too slow, an alternative way to see these slides would be to go directly to my LinkedIn profile to see these and some other slides I have put there.
The 87th MPEG meeting took place at EPFL in Lausanne, Switzerland, from February 1st to 6th, 2009. This was the 7th time MPEG committee was coming to Switzerland at the invitation of the Swiss National Body to ISO, and the third time the committee came to EPFL. This meeting was very productive with many exciting developments. Among others, one can mention activities around 3D video coding, and the newly launched High Performance Video Coding, referred to as HVC, which aims at creating the next generation video compression algorithms beyond Advanced Video Compression AVC (also known as MPEG-4 part 10 and H.264). Another highlight of the meeting was the announcement that MPEG committee has received two Emmy Awards thanks to its outstanding contributions with AVC.
Some pictures of the meeting can be consulted at the following URL:
The forth edition of international workshop on video processing and quality metrics for consumer electronics (VPQM) took place in Scottsdale, Arizona, on January 15 and 16, 2009. This event has now become a traditional gathering place for experts from around the world active in the field of video quality assessment. More details about the forth edition, and previous workshops, together with program and papers presented are available at the following url:
During this workshop I presented a work we did jointly with FUB on the influence of color information in the perception of quality in high resolution pictures. The pdf version of the paper can be accessed at the end of this article.
I took a few pictures while at VPQM, that you can consult here:
It is worth mentionning that I am organizing a workshop related to this topic in San Diego from July 29-31st, 2009, called QoMEX.
Have a look at the following url and consider submitting a contribution:
The now traditional MMSPG 2008 Christmas dinner took place on Saturday December 13th, 2008, at the usual place: Relais de la Poste, a nice restaurant on the easterns hights of Lausanne, in the middle of vineyards. This was a joyful and friendly gathering to meet with colleagues and their family members, and to celebrate a rather dynamic year 2008. Some photos of the dinner can be found at the following url:
The 9th, and probably last lunch meeting of our research group in 2008, took place on November 7th. This lunch was organized by Jong-Seok LEE. Thanks to his wife's kind efforts and contributions, we could all enjoy some interesting and very tasty Korean food and drinks. Our appreciations go to both of them, and we look forward to have another similar experience very soon.
There are some pictures in the following URL of the event, and especially a few snapshots of some of the food we had: