WPF Imaging

preview
Posted by Charles // Mon, Mar 6, 2006 4:24 PM

Recently, Tim Sneath, a Channel 9 correspondent and Technical Evangelist took us over to the WPF imaging team to learn about some of the great work that's been done in that area. Substantial (and innovative) improvements have been made to the way Windows manages images and renders images and the WPF People have surfaced these improvements for us, the developers. There's been great work done in the WPF imaging APIs making it much easier to powerfully maniuplate images from managed code. Check this out.



Show: Going Deep

Tags: ,

Video Length: 00:42:56 Replies: 24 // Views: 16,197
  Zeo
  Channel 9 :)
 
  Mon, Mar 6 2006 9:16 PM

Nice to see the avalon team back on C9.....the more Vista technology videos the beter . Peggy is really cool. She's seems like a perfect candidate for the WM_IN series....but alas seeing her here on C9 is just as cool.

I love the phone ringing in the video. Another person getting voicemail because of C9 filming....hehehehe.

Charles nice camera work....although during parts of the video I got dizzy with all of the back and forth and round and round with moving between all of the people being focused on.



  Minh
 
 
  Mon, Mar 6 2006 5:13 PM
Interesting idea about matching the pattern of bits to a file codec. I guess the 4 CC, magic number isn't used any more?


  BryanF
  Free as in time
 
  Mon, Mar 6 2006 8:21 PM
Great video. Four quick questions:

(1) Are there any analogous WPF subsystems for other types of media (audio, video)?

(2) On a related note, what about vector images, such as SVG, WMF, or Visio diagrams? (SVG in particular)

(3) What's the installation policy with image codecs? The video mentioned than unknown image formats are passed to the application as a raw blob, but is there any service that will attempt to fetch codecs from the web, like in WMP today? Are there any special security issues when installing 3rd party codecs?

(4) Not for the imaging team, but will IE7 be upgraded to this new API? It would be great to future-proof against new codecs to avoid another wait between releases as with transparent PNGs.

Much thanks.

  DigitalDud
 
 
  Mon, Mar 6 2006 9:11 PM
You guys need to resurrect JPEG 2000. It was so much better than any image format we have today, beautiful images, great compression compared to JPEG, loss-less mode, it was awesome. Yet it never seemed to catch on, I guess because it was so complicated.  Wavelets are definetly superior to cosine transforms.

  alan_burchill
 
 
  Tue, Mar 7 2006 6:07 AM
What was the url for the guys blog where he post the WIC Explorer...??

Thanks

Alan

  Sepcot
 
 
  Tue, Mar 7 2006 10:10 AM
alan_burchill wrote:
What was the url for the guys blog where he post the WIC Explorer...??

Thanks

Alan


The WIC Explorer code is up on Robert Wlodarczyk's Blog: http://blogs.msdn.com/rwlodarc/archive/2005/09/21/472338.aspx


  RWlodarczyk
 
 
  Tue, Mar 7 2006 10:24 AM
Indeed, that's correct. I haven't updated it for the Feb. CTP yet, but will be doing so in the next few days.

  PerfectPhase
 
 
  Tue, Mar 7 2006 12:59 PM
With all this plugable codec options, is there going to be a way to select which is the default for an image type.  I can imagine several different apps installing their own codecs for types like RAW.  I find it a pain with DirectShow to change the priorty of filters registered for the same type, some work much better than others

  SlackmasterK
  Its like when monkeys use tools for the first time
 
  Tue, Mar 7 2006 1:16 PM
must... install.. motors on desk...
cannot... trust... weight of equipment... to small EM currents...
therefore must remember locknut.

  rasx
  Programmer/Analyst III, Emperor of String.Empty
 
  Tue, Mar 7 2006 3:50 PM

This Blog entry will talk about Channel9 Videos. This completes my disingenuous exposition.

Apart from Robert Scoble’s famous tension-filled laugh (which by the way is “socially acceptable”—it just sounds tense to me—but then, again, I like “free jazz” and Sponge Bob does not), there is the continual request to not look in the camera. This advice seems strange to the philistine children of the Reagan era—especially the “technical” folks at Microsoft. But my lack of formal education in media history suggests to me that during the 1980s, the rise of the “infomercial” introduced this innovation. This is a professional camera technique that manipulates the psychological powers of the viewer.

When the subjects in camera do not look at the camera, they suggest to the viewer that they are, at best, part of the conversation—at worst (which more often) the viewer is encouraged to indulge in voyeuristic conquests just short of the pornographic experience. When the talking heads do not look in the camera, the viewer can feel like they are uncovering documentation instead of being subject to a presentation. This technique seduces the viewer into thinking they are in charge of the situation (or at least genuinely respected) when (more often) they are not. The reason why I say they are not is because there are more infomercials in “popular” media than documentaries. The arithmetic does not lie. The camera often does.



  LaBomba
  Does that come with 2.0?
 
  Tue, Mar 7 2006 6:34 PM
 Yes,

going deep... keep them coming guys.

Good stuff.

-LB

  ShadowChaser
  It's not easy programming with paws.
 
  Tue, Mar 7 2006 8:33 PM
BitmapFrame?

I must be dreaming... managed support for image animations designed right into the core of the new imaging classes, and not an afterthought? WOOHOO.

  PeggiG
 
 
  Tue, Mar 7 2006 9:24 PM

Hi Bryan. I've inserted answers to your questions inline. Thanks for asking! - Peggi

BryanF wrote:
Great video. Four quick questions:

(1) Are there any analogous WPF subsystems for other types of media (audio, video)?

   [PG] You can use video and audio in WPF, and even integrate it with animations, images, and user interaction, but we don't have an extensibility platform for it like we do for images.

(2) On a related note, what about vector images, such as SVG, WMF, or Visio diagrams? (SVG in particular)

   [PG]  The only discovery mechanism we have for 3rd party formats currently is for images. This is largely driven by the need to support Raw image formats, because new ones are created fairly frequently. (Of course, it isn't limited to Raw formats, but it is currently limited to image formats.)

(3) What's the installation policy with image codecs? The video mentioned than unknown image formats are passed to the application as a raw blob, but is there any service that will attempt to fetch codecs from the web, like in WMP today? Are there any special security issues when installing 3rd party codecs?

   [PG] When you install an image codec, it gets registered in the registry and we match a pattern in the image file against the registry entry to identify the codec when an image is requested to be loaded. If the image format is unknown (no matching codec on the machine), today, we just return an error saying the format is unknown. (It's actually unknown metadata formats that we treat as a BLOB; that wouldn't really work for image formats.)  

   - Having a service that attempts to retrieve a codec from the Web would be a good idea, and one that we'll consider for future versions. We do need to be careful about installing 3rd party codecs because of the security risks. It's one thing for a user to consciously decide to install a 3rd party codec, but if we (Microsoft) install it, we have to have a way to make sure it's really secure. Otherwise, we'd be putting our users at risk. 

   - There are always security risks when installing a codec. We've gone to great lengths to make the codecs we ship with WIC/WPF as secure as possible, but we currently do not have a way to ensure that 3rd party codecs are secure. That's why they require admin priveleges to install them. That's not really a solution, though. In the future, we plan to have a certification program for image codecs that will require a specified number of hours of "file fuzzing" for any 3rd party image codec to ensure at least a baseline of security.  In the meantime, if an image codec is discovered to have a security vulnerability, we will add it to a blacklist that we check before loading any codec. Blacklisted codecs will never be loaded. The codec manufacturer will have to release a security patch to reenable their image format if this occurs.


(4) Not for the imaging team, but will IE7 be upgraded to this new API? It would be great to future-proof against new codecs to avoid another wait between releases as with transparent PNGs.

   [PG] Not being on the IE team, I can't make any promises, but I expect that they will be using the Windows Imaging Component. I just don't know what the timeframe will be.

Much thanks.



  PeggiG
 
 
  Tue, Mar 7 2006 9:35 PM
PerfectPhase wrote:
With all this plugable codec options, is there going to be a way to select which is the default for an image type.  I can imagine several different apps installing their own codecs for types like RAW.  I find it a pain with DirectShow to change the priorty of filters registered for the same type, some work much better than others


   [PG]  It would certainly be possible to provide a way for the user to select the default codec for an image format. In general, we don't expect there to be alot of different codecs for the same image format, but it's certainly possible. 

In the developer platform itself, we don't expose any UI to the end user, but an application (or the Windows Explorer) could expose some UI to enable the user to select a default codec for an image format, and then always request that specific format for that file type. That feature isn't supported in Windows Vista but, if enough people request it, there's no reason it couldn't be supported in a future version.

  ibrahim
 
 
  Wed, Mar 8 2006 4:31 AM


  footballism
  Volunteer .NET Evangelist
 
  Wed, Mar 8 2006 6:36 AM
Does any one know where I can get the managed demo app shown in this video?

Sheva


  PeggiG
 
 
  Wed, Mar 8 2006 2:32 PM
Right now, I just have it on my machine.
I'll see if Robert can post it on his blog.

  JoshRoss
 
 
  Thu, Mar 9 2006 12:14 AM
Near the end there was talk about 32bit channels.  Those would be some large files.  An image 8mp would be 72mb raw files.  I would like to see the undo stack push and pop those!

  geekling
  I am an artist
 
  Thu, Mar 9 2006 12:25 AM
[PG]  The only discovery mechanism we have for 3rd party formats currently is for images. This is largely driven by the need to support Raw image formats, because new ones are created fairly frequently. (Of course, it isn't limited to Raw formats, but it is currently limited to image formats.)

Is this discovery mechanism exposed to developers not inside Microsoft (like me?). If so, is there any API documentation about how it works?


  Hasenoehrl
 
 
  Thu, Mar 9 2006 3:45 AM
Is there a way to load a decoder without a pattern?
We currently use the compound document file format for our image.
We have a root CLSID in there. So is there a way to load the decoder only by file extension or a compound file root CLSID?

  billh
 
 
  Fri, Mar 10 2006 9:20 AM
Thanks for this video, Charles...these are the kind of videos that make me reconsider applying to Microsoft (yes, I admit, I've done it before, so sue me ).  I love looking at and putzing around with file formats, and know a little about the various image formats out there.

With regards to "anti-fuzzing", my concern would extend well beyond image files, though...are you doing this for .wav formats, media formats and other types of files?  I know with video streams the Media Player will usually balk in some way (with some type of "corrupted file" error dialog).  It is ridiculously easy to come up with your own file formats, and as an extension of that, fiddle with the ones that are out there now.

Edit: Are there any updated graphics format pages anywhere on the internet? Here is a page of older formats if anybody is technically curious (circa 1997):

http://www.dcs.ed.ac.uk/home/mxr/gfx/2d-hi.html

  RWlodarczyk
 
 
  Sat, Mar 11 2006 4:06 PM
  PeggiG
 
 
  Sat, Mar 11 2006 4:41 PM
geekling wrote:
[PG]  The only discovery mechanism we have for 3rd party formats currently is for images. This is largely driven by the need to support Raw image formats, because new ones are created fairly frequently. (Of course, it isn't limited to Raw formats, but it is currently limited to image formats.)

Is this discovery mechanism exposed to developers not inside Microsoft (like me?). If so, is there any API documentation about how it works?


[PG]  There's no API required. We do the discovery automatically. You just have to include a pattern in the image file that uniquely identifies the image format, and register the pattern in the registry. The discovery process itself is automatic. At runtime, we match the pattern in the file with the pattern in the registry, and that's how we discover the right codec.

BTW, I said that we only use discovery for image formats. That's not quite true. We also use it for metadata handlers and pixel format converters. (But only for images.)

  PeggiG
 
 
  Sat, Mar 11 2006 4:45 PM
Hasenoehrl wrote:
Is there a way to load a decoder without a pattern?
We currently use the compound document file format for our image.
We have a root CLSID in there. So is there a way to load the decoder only by file extension or a compound file root CLSID?


[PG]  We do require a pattern that uniquely identifies the format, but the pattern can be anything you like, as long as it always apprears at the same place in the file.  In fact, we strongly recommend that people use GUIDs for their identifying pattern, because that guarantees the pattern is unique. So, if the CLSID appears at the same place in each file, you can use the CLSID as the pattern.