Windows 8 has been in the market for six months now, offering a logical enough time to evaluate the progress Microsoft has made in that time. After all, the firm last year redefined itself as a maker of devices and services, and Windows 8 fits nicely into both of those categories. How is it really doing?

The icing on this cake, so to speak, would be an official announcement about Windows 8 license sales. Microsoft surprised me by not offering this number in its earnings announcement last week, so I can only assume that it will do so sometime in the months ahead, perhaps at or before TechEd in early June. I’ve said it before, and I’ll say it again: If this figure is any lower than 20 million units per month—or a total of 120 million units through the end of April 2013—Windows 8 is in serious trouble.

But we don’t know about that. Yet. So let’s look at what we do know.

Windows 8: The mile-high view. Windows 8 is a transitionary OS, designed to drag Windows (and Windows users) kicking and screaming into what Microsoft (probably correctly) sees as the future of mainstream personal computing: Highly mobile and connected devices like tablets. As such, it combines the now-legacy desktop environment from previous Windows versions with a new mobile OS I call Metro, and it does so in fairly ham-handed ways. Critics have rightfully called the transition between these two environments as “jarring.” Windows 8 is indeed a FrankenOS.

Initial software quality. As a hybrid OS, it’s a bit hard to evaluate the quality of the initial Windows 8 release fairly, but what the heck: I think the desktop side offers a tremendous upgrade value over Windows 7, and is in fact a bigger upgrade than Windows 7 was over Windows Vista. That this is overshadowed by its being melded to Metro is of course unfortunate, and not just because that melding is artificial and poorly done. Metro is very clearly a 1.0 release that is in desperate need of updating, and given Microsoft’s recent moves into “devices and services” and its plans to upgrade traditional software like Windows and Office as if they were online services, one might understandably believe that the firm would move quickly to address concerns. This has not happened. More on that below.

Windows RT. Windows 8 is notable for many reasons, but one that should be called out is Microsoft’s controversial decision to suddenly resuscitate the platform’s multi-platform roots and issue a version of the OS that runs only on the mobile-oriented ARM platform. The initial release of Windows RT was lackluster at best, and the blame can be spread to many parties, including Windows itself (the Metro environment that is at the heart of Windows RT is, as noted above, a 1.0 release), the almost non-existent (at launch) apps ecosystem, the terrible Xbox-branded media ecosystem, and underperforming current-generation ARM hardware. If it weren’t for the Surface (see below), I’d add “dearth of hardware choices” to that list too, but let’s face it, if you’re going to get a Windows RT device, you want Surface.

Deployment as a service. While Windows 8 is indeed a traditional software product that can be purchased on DVD if you’re a masochist, one of the big changes with this release is its availability online for purchase and install. That is, Microsoft has made Windows 8 available to consumers from the web. And that web installer doesn’t just work, it’s the single best way for individuals to install the product. This was well done, though the cancellation of the initial promotional pricing was not: The Windows 8 Upgrade is simply too expensive.

Servicing as a service. Microsoft has yet to communicate a formal upgrade strategy for Windows 8, as it has done for Office 2013. (That product will be updated quarterly with new features and bug fixes.) Worse, the product hasn’t been updated in any meaningful way since general availability though the majority of the built-in Metro-style apps finally started getting substantial updates over the past 30 days or so. This is an area that needs dramatic improvement, and Microsoft’s plans for a big-bang GA+1 upgrade that’s codenamed “Blue” explains everything you need to know about this team’s inability to deliver constant evolutionary updates: It still has that major milestone mindset. And that’s too bad.

Surface. Despite a lot of griping over both Surface RT and Surface Pro, these products are an undeniable success story for Microsoft and are well on their way to being a major new business for the company. The Surface tablets offer high build quality, are beautiful to look at, and have immediately established Microsoft as an innovative and thoughtful hardware maker. Like many of you, I’m eager to see improved and new designs. But the existing Surface devices are already excellent, and Surface RT is the only Windows RT device that anyone should consider buying, especially when you factor in the monthly firmware improvements: Unlike with Windows 8, Microsoft is serious about continually improving these devices.

Third party hardware. Microsoft was hugely disappointed with the lineup of devices that its hardware partners offered at launch last fall, and with good reason: It was made up of mostly non-touch, traditional PC form factors. Today, things are improving, and the mix of touch-capable devices is getting more diverse. It’s still unclear whether Windows 8’s hybrid PC model will ever completely captivate customers, but then the first generation devices have unfairly tilted my opinion to the negative. It will get better.

Apps. While the initial supply of Metro-style apps for Windows 8/RT was undeniably paltry, the situation has improved rapidly, from both quality and numbers perspectives. I’m going to give this one a pass: The selection is getting there.

Looking forward, Microsoft needs to address several key aspects to the Windows 8 ecosystem. The product itself is titled far too much to consumers and is openly hostile to the firm’s bread-and-butter corporate customers. The Metro design is questionable, but also immature, and needs to be updated rapidly (something we’re finally starting to see in “Blue”). There’s no evidence of better desktop/Metro interfaces, but that needs to happen, as do further improvements to the desktop side of the fence.

Put simply, I don’t believe that Microsoft has ever shipped a version of Windows as incomplete/undone as is Windows 8 and yet the firm is not moving as quickly as I’d like to fix the very glaring issues with this release. One imagines the Windows team believing that yearly updates is faster than once every three years, and thus they’re on the path to success. I don’t think that’s the case: This first year is a crucial one for Windows 8, and aside from a few positives—a smattering of Metro app updates, the Surface firmware updates, and the web-based deployment options—they have essentially squandered the first six months of that time period.

Granted, we’re about to enter a period of rapid change. By June, we should have a “Blue” public preview that speaks completely to what Microsoft plans to do, and in June/July other updates, like a complete revamping of the Xbox media services and apps, will occur as well. But making customers wait until the summer following the initial release is irresponsible and the Windows 8 report card is decidedly mixed as a result. If this was my kid, he’d be heading to summer school.