Microsoft Opens Up the PST Format

From Microsoft:

As more and more information is stored and shared in digital formats, the ability for people to reuse their data across various applications and platforms has become increasing important. As part of an ongoing effort to enable this kind of data portability, Microsoft is announcing that it will be releasing documentation for the .pst file format – the format in which data is stored in Microsoft Outlook Personal Folders.

Providing access to the documentation will facilitate interoperability, enabling customers and vendors to access their data in .pst files across a variety of platforms. This is important to organizations that need exchange key corporate data in and out of Outlook, upload to the cloud, or comply with corporate governance policies.

When it is complete, the documentation will be released under our Open Specification Promise, which will allow anyone to implement the .pst file format on any platform and in any tool, without concerns about patents, and without the need to contact Microsoft in any way.

Paul Lorimer, the Group Manager of Microsoft Office Interoperability has posted additional details on the Interoperability @ Microsoft blog.

Discuss this Article 63

RobertC
on Oct 26, 2009
This should make life much easier.
Ocean
on Oct 26, 2009
ggolcher: Only if you do
rr0de74@live.com
on Oct 26, 2009
I am not sure why MS is doing this. It will just help the likes of Google and many others migrate off of Exchange. Lots of Mac switches fight with PST files. This move will make it easier to switch.
Delmont
on Oct 26, 2009
Paul, Enough is enough. Please bounce Ocean from your blog. At least for a month to teach him a lesson. He and the other trolls are ruining your web page!
anonymous
on Oct 26, 2009
Data portability has become an increasing need for our customers and partners as more information is
NoNameAtAll
on Oct 26, 2009
It wouldn't be a problem. But Ocean's links have nothing to do with the PST format or Outlook. Thus, it is off-topic and does not belong here.
roteague
on Oct 26, 2009
Great news, but I bet it is as convoluted as the Excel format. I'm willing to give it a look.
Waethorn
on Oct 26, 2009
"Paul refuses to blog about any negative Microsoft-news" There's plenty of Apple sites on the internet that do that already.
nutmac
on Oct 26, 2009
While I am glad PST format is finally open, I want to chime on how horrible I think this format is. A single large file that stores every Outlook data is just plain bad idea. Difficult to backup and prone to corruption.
anonymous
on Oct 26, 2009
This post was mentioned on Twitter by Kittyburgers: RT @EverythingMS: Microsoft Opens Up the PST Format http://bit.ly/45priv
Delmont
on Oct 26, 2009
nutmac, I don't understand your point. Do you want to store each folder into a seperate pst file in Outlook? You can configure Outlook to do this, but why give yourself all that extra work? Why not just archive your email once a year or 6 months and start a fresh pst file?
techman.merb
on Oct 26, 2009
nutmac, backing up pst files is about the easiest thing possible. Just download the pst backup tool from Microsoft and you can configure Outlook to backup the pst to whatever location you want, whether a local or network drive. I've never understood why they didn't include this tool with Outlook because most people are not even aware that it exists.
Waethorn
on Oct 26, 2009
"Difficult to backup and prone to corruption." Would you rather have multiple files in XML format with zero security, each mail message being uncompressed, and in multiple folders? I have never run into a problem where a PST file became corrupt where I couldn't restore from a backup (Shadow Copies worked to solve the problem), or run the PST cleanup tools. I've run into a few issues where PST files became too big though, but that's due to users not archiving messages properly, or holding onto messages with attachments, where the attachments were already saved to disk.
Waethorn
on Oct 26, 2009
"backing up pst files is about the easiest thing possible. Just download the pst backup tool from Microsoft and you can configure Outlook to backup the pst to whatever location you want, whether a local or network drive." I just let SBS Backup handle that - it backs up Exchange mailboxes along with every thing else.
Waethorn
on Oct 26, 2009
"I want to people here to see the bad and the good." Too bad - not your blog!
shark47
on Oct 26, 2009
"I want to people here to see the bad and the good." Why not troll about politics and movies while you're at it.
gadfly10
on Oct 26, 2009
Great. Now how about opening up that useless 2GB limit in exmerge.
Logjamming
on Oct 26, 2009
Hey...how about that W7 upgrade/clean install stuff? The new Getamac adds are spot on! Broken promises, broken promises, broken promises. Oh, and copied software, copied software, copied software. Oh, and lawsuit, lawsuit, lawsuit. It's 2005-6 all over again for Microsoft. Advertising, advertising, advertising, copy OSX, advertising, advertising, advertising, copy OSX, advertising, advertising, advertising, copy OSX.
rr0de74@live.com
on Oct 26, 2009
"Apples lackluster graphics options in the new Mac lineup"???? Oh I get it it wont play PC games at uber/l33t FPS when booted into Windows. For anything you would do in OS X the video card options are just fine. PC gaming is dead Wae, did you not get the memo?
Ocean
on Oct 26, 2009
Movies? Transformers 2 comes out tomorrow on DVD...Amelia looks bad...but Where the Wild Things Are looks good....
rr0de74@live.com
on Oct 26, 2009
"Too bad - not your blog!" Not yours either. Stop him from posting, I dare you!!!!!!
lotsamystuff
on Oct 26, 2009
"Migrating is easy when you throw away an application" http://community.zdnet.co.uk/blog/0,1000000567,10014289o-2000498448b,00.htm
whiplash55
on Oct 26, 2009
@Ocean WTF are you talking about? Is this a post about Win Mobile? Loggamin never heard Paul go off on Win Mobile so he can be excused Ocean's been around a while and should know Paul criticizes MS frequently when he thinks what they're doing is senseless (multiple versions of Windows) or just lousy as in Win Mobile. Opening up the PST format is a good thing, by the way Paul hates Outlook so you pathetic Mac drones can relax for once, he prefers Gmail or Windows Live.
runner7775
on Oct 26, 2009
"PC gaming is dead Wae, did you not get the memo?" I think 11.5 million WOW subscribers would beg to differ. Not to mention around 20 million Steam users.
whiplash55
on Oct 26, 2009
The Getaamac ads are stale and as usual short on truth. The upgrade media did a clean install and activated with 0 issues. Not that Lojammin has ever not had his head as far up his arse as is possible, but I thank him for demonstrating it yet again.
pthurrott
on Oct 26, 2009
OK. So, I've obviously been busy lately. Ocean, you're about to be banned. This time it will be for good. Off-topics posts are not allowed. I will obvioulsy have to treat you like the child that you behave like. This is not the place for this nonsense. Enough of a warning? Next time, I just pull the plug.
Ocean
on Oct 26, 2009
"Enough of a warning?" Sure.
rr0de74@live.com
on Oct 26, 2009
Yeah very true about Wow, but WOW is 85% of PC gaming. I read an article recently that said COD4 sold 14 million copies, less than 600,000 were on the PC. Sad.
roteague
on Oct 26, 2009
Thanks Paul. It would be nice to discuss some aspect of Windows, without having to wade through all the Windows hate that a few seem determined to throw in our way.
runner7775
on Oct 26, 2009
@rrode Yeah that is sad, COD4 on PC is excellent. I just read that, according to a year and a half old study, that WOW has 62 percent of just MMO's. I still am of the opinion that PC gaming is far from its last breath, but I (sadly) expect many of the best PC games will continue to be ported from the consoles(not developed PC then sent to the consoles).
Ocean
on Oct 26, 2009
btw...there are a couple of other OT posts here. I've pointed them out below so everyone can see whether you are just a child bullying another 'child' or whether you truly have standards. I'm betting it's the latter. @5:45 @5:05 @4:43 @4:18 @4:15 @3:46
Backup77
on Oct 26, 2009
@techman.merb You make a good point, most people are unaware of the PST backup tool and why this is just not included with Outlook is bizarre. I do find it easier to back up Outlook files than Outlook Express which is a mess.
nutmac
on Oct 26, 2009
I wasn't aware of PST backup tool and frankly, I wouldn't use it. I like to use built-in backup tool (or something more standards based) to backup the entire drive. PST file has a tendency to become huge. Telling users to delete unneeded messages and/or archive is an awkward solution. The problem is exasperated by the fact that most Outlook installations default to older 2002 format, which maxes out at 2 GB. Most other PIM/email clients (e.g., Evolution, Apple Mail) store data across multiple compressed files and/or directories. They are much more incremental backup friendly.
rr0de74@live.com
on Oct 26, 2009
Yeah it used to be PC games were ported. Now it the other way around. Setting most PC games above 1280x720 is doing nothing as that is the max detail setting you are ever going to get out of it because its what the console game are coded for. I remember PC games with lots of sound options for EAX and other things, most all gone now. However Infanty Ward the makers of COD4 stated flat out that piracy could stop them from making PC games. Ocean there is not justice on this site. People tear each other up yet Paul picks on you. Sad. Get banned, come back as Dirt or Air, or "Paul's Mama". Better yet if he bans you, go on Amazon make 50 fake accounts and give his book 1 star:)
evgenij
on Oct 26, 2009
"Ocean there is not justice on this site. People tear each other up yet Paul picks on you. Sad. Get banned, come back as Dirt or Air, or "Paul's Mama"." This is Paul's blog, not yours. He can do whatever he wants. He can write about Windows, he can write about Apple and he can ban Ocean - it is all up to him because it is HIS site. Thank you, Paul for finally cleaning up this place. Too many gremlins lately.
Delmont
on Oct 26, 2009
I think people have to remember Paul is a real person. And this is his site. This site doesn't belong to some faceless corporation. It's Paul, a real person. So relax on him. Personally the comment posted at 7:42pm is also very out of line. People, we come to discuss computers and software. That's all. This behavior of getting personal and references to actually hurting someone's income is way way out of line! Please, grow up, stay on topic or simply leave. It's that simple.
robertsjoe
on Oct 26, 2009
+1 to ban @ocean
lketchum
on Oct 26, 2009
good to see them open it up; however, PST is not at all what most Outlook on Exchange users run at all. Nearly all (this is the default, too) run an .OST file, which is a local cache of their mailbox on the Exchange. Even where internal/LAN clients connect directly (as opposed to using Outlook Anywhere via RPC over HTTPS), have a locally cached copy in apps data and again, it's an OST file. Very few people also create a local PST and drag items out of their mailbox into personal folders (stored locally only and in addition to Exchange Server side and local OST cache's). Given this much more common reality, it (to me) takes away from the significance of this release.
lketchum
on Oct 26, 2009
@nutmac Since Outlook 2003, local PST files in Unicode-8 format have storage potential of 16 Terabytes (yes Tera with a "T"). Only in such cases where an existing PST on older version was upgraded, without being converted, would the 2.2 GB limit apply. OST, Like 2003 a newer versions (as well as any single Exchange Store) may be up to 16 Terabytes. Our customers routinely exceed 10 GB Exchange mailboxes and > 15 GB is very common. Outlook and Exchange were designed to handle very large email client sizes and equally busy users. If managed well, users can enjoy trouble free email for years without offline maintenance.
rr0de74@live.com
on Oct 26, 2009
An OST is a PST that is locked/tied to an Exchange/AD account and usually used only for offline access of a mailbox. As soon as you turn on cached mode in Outlook a OST is created. If I gave you a OST and PST on a external drive you could open the PST with a copy of Outlook. The OST would not open. You can get special tools that will break the OST and basically turn it into a PST, chop off the security feature. Lots of users will have PST files if their Exchange mailbox has a limit. The create PST files to move stuff off the server to not hit their limit. Also if Outlook Archiving is turned on the Archive is a .PST file. For home users that are lame enough to use the full Outlook for a POP3 or IMAP account then all of that data is stored in a .PST file. PST file are everywhere.
lketchum
on Oct 26, 2009
@rrode74, You said: "As soon as you turn on cached mode in Outlook a OST is created." Rrode, cached mode using an OST vice a PST is the DEFAULT configuration. Outlook is configured this way unless the user/administrator intentionally modify it. Exchange ships with defaults limiting the size of mail boxes and administrators adjust these limits according to their needs. They are set globally, but may be adjusted for individual users and groups, or the global policy may be adjusted as needed. PST files have not been common for Outlook clients connected to an Exchange server for many years. Further, Outlook creates archives per machine and within the mailbox folder - not in a PST. The theoretical OST file size is 16 Terabytes and uses the same object database type as the Exchange and its mail stores do - which is far faster than a relational database type. OST files may be removed, incremented to create a new one, and moved, migrated and managed using tools freely available to Exchange administrators and others/end users. There are also many (costly, but effective) commercial third party tools used to manage Exchange mailboxes, and typically they are used against the mailbox stored on the Exchange Server. Very often, home users of Outlook connect to a hosted Exchange provider, including Microsoft's own hosted services and in such cases, a PST is not created or used. Since August of 2003, RPC over HTTPS has been used to support an experience identical to being on a local LAN. This evolved to Outlook Anywhere with Outlook 2007. Prior to August 2003, remote, un-joined Outlook users (2000 and 2002/Office XP) could connect to an Exchange server without a VPN by using forced encryption over TCP - provided the Exchange server was published via Microsoft's Internet Security and Acceleration (ISA) Server. My company has built, hosted and supported these services since 2000. Your comments do not reflect a credible understanding of these technologies or the familiarity that comes from working with them on a daily basis. An OST file is not used "ONLY" to support offline folder access. It is the default and used to support localized caching that benefits from late-start capabilities that are rules based and network aware. A large number of variables are continuously monitored and factored and in ways that are most often transparent to end users, message delivery is streamed to the client and its localized object database as it polls "with" the server - even aggregate server load and policy based delivery rules are applied (higher priority traffic moves first, for example). Provided the context name and security descriptor were satisfied (e.g., the OST was being opened by an authorized party within the Exchange Organization), one could open it as easily as they could a PST - the difference being the protection afforded to Exchange mailboxes (which is how organizations and I hope end users, would want it to be). Outlook is an exceptionally good communications and PIM client, which can only be fully appreciated when used with Exchange. All of the other "me too" alternatives do not come close to its functionality. When used with other member servers, like SharePoint and apps data, which is easily exposed to custom and other commercial applications, it is very difficult to match its capabilities.
rr0de74@live.com
on Oct 27, 2009
So you basically repeated what I said, but it took you many more words?? You started off by quoting me, then saying the exact same thing I said???? Cached mode creates an OST not a PST I am glad we both agree. If you DONT go into Cached mode with a Outlook profile an OST WONT be created. You said... "An OST file is not used "ONLY" to support offline folder access. It is the default and used to support localized caching that benefits from late-start capabilities that are rules based and network aware. " Yep the default setting when you put the Outlook profile in Cached mode. Cached mode is an improved offline mode that came with Outlook 2003 and Exchange 2003. It detects your network status and acts accordingly. Prior to cached mode you were either in online mode (not OST) or offline mode where you lived out of a OST and either set the sync to a specific interval or invoked it manually. You can setup Outlook Anywhere to NOT use an OST if you want, by NOT turning on Cached Mode. You dont need ISA server to provide encryption, its provided via the SSL cert. ISA adds another layer and allows you not to directly expose your Exchange server. It bridges the SSL link and the user really never gets past the ISA server, but its NOT required for encryption, only an SSL cert is required. If I gave you my OST you could not open it, since my AD/Exchange properties are tied to it. You would need a third party to tool to crack it. If I gave you my PST (not password protected) you would be able to open it and read it with just Outlook. Exactly what I said the first time. I have been supporting Exchange since version 4.0 Outlook/Exchange are very good business collaboration tools, and VERY expensive.
Waethorn
on Oct 27, 2009
@rrode: OpenCL on Mac is now a joke, because their graphics card options are a joke (especially on the Mac Pro - ie. only one half-decent consumer card, but no pro cards LOL!). And what happened to their alliance with NVIDIA? All the Mac's with new internals are now back on Intel chipsets, which Apple previously slammed for having poor power management. That's two partners that Apple has recently b*tch-slapped. One for Intel, and another for NVIDIA. Where's your Apple apologies for that? Anyway, opening up PST means that interoperability between different email clients will be easier. Moving from a peer-based client setup to Exchange will also be much easier if the desktop client supports export to PST. Importing PST's into Outlook for use on Exchange is extremely easy. Of course, you can do it on the server end too.
Waethorn
on Oct 27, 2009
"If I gave you my PST (not password protected) you would be able to open it and read it with just Outlook." Only if you choose the no password option. You can certainly protect PST files with encryption.
Waethorn
on Oct 27, 2009
"Our customers routinely exceed 10 GB Exchange mailboxes and > 15 GB is very common. Outlook and Exchange were designed to handle very large email client sizes and equally busy users." I don't care who you are - 15GB is a bit insane for an Exchange mailbox. Large files are best stored elsewhere on the system, not left as attachments in a mailbox. I think I read that Exchange 2007 SP1 increased the supported mailbox size to a whopping 75GB or something. That's just ludicrous in my book. I'd hate to have to see a new Outlook client synchronize an offline copy of that.
Dipsh t Admin
on Oct 27, 2009
"I don't care who you are - 15GB is a bit insane for an Exchange mailbox." I agree. You need to look at some other options if it's getting that big. It can take long enough just to sync 1 GB, I can't imagine what more than 10 GB would be like.
Delmont
on Oct 27, 2009
Waethorn said: "Our customers routinely exceed 10 GB Exchange mailboxes and > 15 GB is very common. Outlook and Exchange were designed to handle very large email client sizes and equally busy users." I don't care who you are - 15GB is a bit insane for an Exchange mailbox. Large files are best stored elsewhere on the system, not left as attachments in a mailbox. I think I read that Exchange 2007 SP1 increased the supported mailbox size to a whopping 75GB or something. That's just ludicrous in my book. I'd hate to have to see a new Outlook client synchronize an offline copy of that. Yes the solution is called Sharepoint :-)
Waethorn
on Oct 27, 2009
"Yes the solution is called Sharepoint" I was implying that. That's a best practise in the SBS world for file storage. Network folder access over VPN is not, due to security issues. Who actually trusts home workers to connect to the business network over VPN? I know I don't. We use RWW exclusively here. Sharepoint, OWA, and Terminal Services Gateway are much safer than connecting an untrusted computer over VPN. Our Remote Desktop sessions are locked down so that the network isn't exposed to software that is running on the user's home machine. SSL security is also stronger than typical VPN, and easier to setup than a complete VPN system infrastructure.
GoodThings2Life
on Oct 27, 2009
I can't believe all the commotion on this... Microsoft is damned if they do, damned if they don't with you fruit-loving freaks! They're opening up a proprietary standard so that others can use it, which will not only mean better recovery/support tools but better cross-platform support. This is a GOOD thing for everyone, and if you don't like the format that's fine too... why the hatred though? A few of you are clearly trolls that will desperately try to destroy any good news Microsoft makes. Some of you claim the Mac ads are spot on, but you're wrong. I can upgrade from XP to W7 a whole lot easier than I can move to Mac. I run the Windows Easy Transfer utility to backup my files and settings... install Windows 7... restore the settings using the same utility, and yes, I reinstall my apps. But at least my apps are going to work 99.999% of the time. Oh you moved to Mac and now I gotta buy a new copy of Office?! New copy of Photoshop/Acrobat/whatever?! WTF-- Application X doesn't exist on Mac?! Who has the easier migration path? Exactly why do you think that Apple has made it so easy to run Windows on a Mac but not the other way around? Hint: It's not because the platform is better, it's because the platform is insignificant. Oh-- the Mac can't run the app you want? No problem, just install Windows in Boot Camp/Parallels/VMware. Microsoft to the rescue-- just like in the 90's with the commitment to Office for Mac and $150 million in "bailout" funds; just like when they made the iPhone business friendly by allowing ActiveSync licensing... well, hopefully by now you get the idea (though I doubt it from the posts I read from some of you). And what's really ironic... you come a site called "Windows Super Site" and expect to see honesty. Well, Paul is actually very good about blogging the good and bad of Microsoft, but when it comes down to it... should I expect Apple Insider to sing the praises of Windows 7 too? Of course not! Here's my real problem with Apple and Macs... they sell through negative advertising. They attack the competition and claim they're better... but they never bother to say why. The few claims they make are half-truths at best and outright lies at worst. Sell your products Apple. Your iPhone are the only good marketing you have.
Waethorn
on Oct 27, 2009
I have a question for anybody that knows the answer: What's the deal with Outlook 2007? I can drag and drop files directly into an Exchange mailbox folder and the file shows up in the message list. What is that feature called? Is that a replacement for Public Folders, which have been deprecated in Exchange 2007? It doesn't seem to work on Outlook 2003, but it works on Outlook 2007. In Outlook 2003, if you drag and drop a file into the window, it just opens a new message and attaches the file. Outlook 2007's behaviour is not the same.

Please or Register to post comments.

IT/Dev Connections

Las Vegas
September 30th - October 4th

Paul ThurottYou'll have the opportunity to experience:
• 120 Technical
Sessions
• Networking with Peers
• Expert Speakers


Come See Paul Thurrott & Mary Jo Foley in Person!

Register Now

Office 365 InfoCenter

Get the latest insight and info from Paul

Read Now!

What I Use