6 things wrong with the Windows Insider program

[Karlston]

Microsoft's Insider Program has fallen off the rails, but a few simple fixes would go a long way

The Windows Insider program has been hugely successful where it counts the most: making Windows fans feel more connected to the product. As a marketing exercise it deserves an A+. But dissension is spreading in the ranks.

 

Most of the problems are inherent in the construction of the Insider program, and some of it's the normal grousing of beta testers. There are a few places, though, where Microsoft could make significant improvements.

 

The Windows Insider program is open to everyone, and guru wannabes have flocked to sign up. At the end of 2014, the program boasted 1.5 million enlistees. Heaven only knows how many are signed up now -- Microsoft isn't saying.

 

People think of the Windows Insider program as a beta testing program for Windows. Not so -- Microsoft has a long history of beta testing programs, both overt and covert, but the Insider program isn't one of them. The Insider program is more of an extended marketing effort, like the "open" beta that appeared at the end of the Windows 7 dev cycle. As I explained in 2009:

 

Microsoft doesn't do open betas -- never has, at least since the days of Word 2.0. All of the open betas nowadays are what we used to call "marketing betas." Microsoft doesn't release betas to the public in order to ask for comments/suggestions about features. Microsoft releases marketing betas so people have a chance to get an early look, convince their companies to wait for a newer and better product -- and in some cases garner bragging rights.

Don't get me wrong. Compared to the Kremlin-like approach of the Windows 8 era, Windows 10 has been a typhoon of fresh air. But there are flaws in the structure of the Insider program, and much could be done to improve it -- even if the program is a marketing effort instead of a technical one. Here are the problems I hear about most often.

1. The wrong people are in the program

Insiders complain rather frequently that they're tired of rubbing elbows with less-than-qualified beta testers. I understand their point of view, but ultimately it's like old geezers yelling at the kids to get off the lawn. Their indignation doesn't worry me.

 

What worries me are the people in the Insider program who don't understand what they've gotten into. When people write in with Win10 problems, I've learned to first ask which version they're running. In many cases, they're in the Insider Fast ring. They've followed advice on the web, seen a discussion somewhere, or simply poked around, and (in spite of copious warnings) signed up -- to disastrous effect. They're floundering with a beta build and don't know what to do.

 

Right now, reverting from a beta version of Win10 to a release version involves reinstalling Windows from scratch. Microsoft needs to create a prominent "bail out" option for beta builds. There should be an easy way to roll back to a prebeta version of Windows 10 without reinstalling everything. If Microsoft needs to keep a prebeta copy of system files and settings in the cloud, so be it.

 

2. The program is so complex even the experts don't get it

This morning I saw notifications all over the web that Microsoft had released a cumulative update for Windows 10 -- when in fact, it had done no such thing. Here's a typical announcement, from a respected industry pundit in a widely read blog:

Cumulative update KB3194496 is now live for those on the Insider Release Preview ring (and likely Slow as well)… The changelog applies to both Windows 10 build 14393.222 for PC and Windows 10 Mobile build 14393.221 released earlier this week.

Another blog:

Microsoft today released a new cumulative update for Windows 10 Insiders on the Release Preview and Slow rings. Build 14393.222 comes one day after the company offered Mobile build 14393.221 to Insiders.

In fact, Microsoft pushed a test version of Build 14393.222 for PCs and 14393.221 for mobile to the Insider Release Preview ring. As best I can tell, there was nothing pushed to any Slow rings. The fact that there's such widespread confusion speaks to the poorly thought-out nature of the "ring" system.

3. Mixing desktop and mobile Insider programs creates endless confusion

Very few people care about Windows 10 Mobile -- and even fewer care about Windows 10 Mobile betas. Yet Microsoft insists on intermingling version numbers and announcing PC and Mobile beta builds without drawing a distinction between desktop and mobile.

 

For example, right now the latest builds for the latest version of Win10 are 14393.187 for desktop, 14393.189 for mobile. The latest Release Preview builds are 14393.222 for PC and 14393.221 for Mobile. Maybe someday we'll get versions of Windows 10 that work in lockstep on both PC and mobile (and Xbox and HoloLens and refrigerators), but that day isn't here yet. In the interim, interleaving build numbers only leads to confusion.

 

I believe that the ongoing "Release Preview and Slow ring" misidentification owes much to exactly this problem.

4. Release Preview ring builds aren't beta builds and need a different name

No doubt it was easier for Microsoft to roll the Release Preview function into the Windows Insider beta build machine, but that commingling has led to endless confusion.

 

On the one hand you have beta builds -- test copies of the next version of Windows. Right now, those builds are numbered 14931, and they'll keep getting larger.

 

On the other hand, you have Release Preview builds -- precursors to the next cumulative update for the current version of Windows. Right now, those builds are numbered 14393.221 and 14393.222.

 

The Windows Insider program caters to both those who are beta testing the next version of Windows and those who are testing the next cumulative updates to the current version of Windows. You bet it's confusing -- and the discrepancy contributes to the "Release Preview and Slow ring" misidentification.

 

Microsoft has used the phrase "Release Preview" for all sorts of items over the years. It needs to give the "precursor to the next cumulative update" builds a different name.

5. We need Release Preview ring builds for all current versions of Win10

Right now, those who wish to test the next cumulative update for the current version of Windows have only one option: The Release Preview ring of the Insider program. When you join the Release Preview ring, you get advance copies of the next cumulative update to the latest released version of Windows.

 

Right now, that means you get an advance copy of the next cumulative update to Windows 10 version 1607. But most of the Win10 world is running on version 1511, and there's no way to sign up for previews of the next cumulative update for version 1511.

 

We saw this scenario play out earlier this month: While admins and devs (and anyone else who's interested) could download a preview of the next cumulative update to version 1607, getting a sneak peek at version 1511 wasn't so easy:

Microsoft has an important "sneak peek" channel for those of you who want to test new builds of Win10 Anniversary Update before they go out to the hoi polloi. It's particularly important for admins and devs who need to make sure that their wares work with the next release. If you need to look into the future just a little bit, make sure you sign up for the Win10 Insider Release Preview ring.

 

Unfortunately, there's no Release Preview available for the Win10 Fall Update, version 1511, which is the Win10 version most people are using. The closest we have right now is a weird preview in the Windows Update Catalog called KB 3186988. Apparently whoever thought up the Insider Preview approach -- it's a great idea! -- didn't think there'd ever be a time when two different Win10 versions would need a sneak peek

Microsoft needs to make the Release Preview ring (or whatever it ends up being called) available for all current versions of Win10.

6. The Windows Insider program doesn't catch even the most widespread problems

Perhaps it's expecting too much from a marketing beta, but I can't fathom how all of the big bugs in the Win10 Anniversary Update evaded detection when it was in the hands of millions of users for weeks before its release.

 

We saw major, pandemic problems with the following:

 

• The installer got hung at 45 percent and 95 percent. It was eventually solved by re-releasing the Anniversary Update, but not changing build numbers
• Bluescreens when Amazon Kindles were plugged in
• The RAW format partition was misidentified, which led many people to reformat perfectly valid drives
• The double-printing bug, which led to a poorly documented hotfix
• The camera driver bug, which led Logitech C920 cameras to clobber Skype

 

The failure of the Windows Insider community -- including me -- to catch any of those big bugs lays bare the fallacy of volunteer testing. The downward spiral started in 2014 when Microsoft fired much of its testing crew and pushed the testing burden back onto its devs. Now Insiders are carrying the ball, and we're fumbling it badly.

 

As Gregg Keizer said in Computerworld early last year:

Windows customers have been griping for months about the quality of Microsoft's updates, many of which have been problem-filled and some of which have had to be pulled because of those problems. Some analysts and users have connected the dots, arguing that after Microsoft's massive layoffs in mid-2014, when the company's software testing groups were especially hard hit, the quality of its updates declined.

 

"Microsoft fired all those testers last year," pointed out Wes Miller, an analyst with Directions on Microsoft, in a Friday interview. "Now consumers are the testers."

We consumers have obviously failed. What does that say about the Windows Insider program?

 

Those participating in the Windows Insider program might see the colour of an icon change in response to their upvoting and kvetching. But real changes? Those are very few and far between.

 

If we couldn't even catch the Kindle bug or the camera driver bug, what good are we?

 

Source: 6 things wrong with the Windows Insider program (InfoWorld - Woody Leonhard)

 

InfoWorld - Woody on Windows

 

AskWoody.com - Woody Leonhard's no-bull news, tips and help for Windows and Office