Jump to content

Ex-Microsoft employee explains quality issues with Windows 10 development


Karlston

Recommended Posts

Ex-Microsoft employee explains quality issues with Windows 10 development

Bizarre bugs and issues with Windows 10 have become rather par for the course these days, and occasionally these can be catastrophic, such as the data loss issue October last year.

 

There have been many explanations for this, with a recent popular one with Paul Thurrott being that the best developers now work in Azure and other cloud ventures, leaving only the least talented to work on a legacy operating system.

 

Recently Barnacules, an ex-Microsoft developer who famously blogged about being fired by Microsoft with 150 other colleagues, posted a video addressing the quality issues a Microsoft, and his explanation is simpler – that Microsoft firing testers in 2015 meant the end of quality software at the company.

 

See the video where he expands on the issue  below:

 

 

 

 

Source: Ex-Microsoft employee explains quality issues with Windows 10 development  (MSPoweruser)

Link to comment
Share on other sites


  • Replies 8
  • Views 1.5k
  • Created
  • Last Reply

I installed Linux based OS to my computers.

Suddenly my computer do what i want not what others enforce remotely.

I was windows user since Windows 3.1 come out, but Windows 10 become a permanent beta software.

Link to comment
Share on other sites


5 hours ago, Karlston said:

Ex-Microsoft employee explains quality issues with Windows 10 development

Bizarre bugs and issues with Windows 10 have become rather par for the course these days, and occasionally these can be catastrophic, such as the data loss issue October last year.

 

There have been many explanations for this, with a recent popular one with Paul Thurrott being that the best developers now work in Azure and other cloud ventures, leaving only the least talented to work on a legacy operating system.

 

Recently Barnacules, an ex-Microsoft developer who famously blogged about being fired by Microsoft with 150 other colleagues, posted a video addressing the quality issues a Microsoft, and his explanation is simpler – that Microsoft firing testers in 2015 meant the end of quality software at the company.

 

See the video where he expands on the issue  below:

 

 

 

 

Source: Ex-Microsoft employee explains quality issues with Windows 10 development  (MSPoweruser)

 

Wondering why in the first place Microsoft fired the testers. Their explanation on firing makes no sense at all.

Link to comment
Share on other sites


3 hours ago, Edward Raja said:

Wondering why in the first place Microsoft fired the testers. Their explanation on firing makes no sense at all.

 

Basically, they thought they could do it as well or better using...

 

Automated testing using Virtual Machines which have little or no hardware diversity (instead of a 1,000 real devices with a range of different hardware)

Windows Insiders unprofessional testers (instead of the professional testers they had)

Telemetry which is insufficient except for application crashes

 

What could possibly go wrong?

 

That was a fascinating video, especially for a (true) insider picture of how testing used to be and why it worked well, and also what their new testing methodology is, and why it has failed.

Link to comment
Share on other sites


Increase of bugs in Windows updated explained

 

73d28b371c45ae65e456b1bd5e4fd6ae_L.jpg

 

Vole decided to fire all the people responsible for bug checking 

 

Software King of the World Microsoft apparently adopted an "anti-vaxer" policy when it came to bugs in Windows updates.

 

Apparently someone in the Vole Hill thought that Windows updates would go away if they fired all those who were expected to check them.  A similar idea is followed by parents who think it is better not to give their children vaccinations because they are not sick. Both strategies have resulted in epidemics. 

 

According to Winfuture,  former Microsoft Senior SDET Jerry Berg, better known as Barnacules, who worked for Vole for 15 years at Microsoft has as one of his roles to design and develop tools and processes to automate testing for the Microsoft Windows operating system. He left the company after Windows 8.1 shipped to the public.

 

Berg said Microsoft's testing processes changed since he worked there. In 2014/2015, Vole employed an entire team that was dedicated to testing the operating system, builds, updates, drivers, and other code. The team consisted of multiple groups that would run tests and discuss bugs and issues in daily meetings. Tests were conducted manually by the team and through automated testing, and if tests were passed, would give the okay to integrate the code into Windows.

 

The teams ran the tests on "real" hardware in a lab through automated testing. The machines had different hardware components, e.g. processors, hard drives, video and sound cards, and other components to cover a wide range of system configurations, and this meant that bugs that affected only certain hardware components or configurations were detected in the process.

 

However Vole laid off almost the entire Windows Test team as it moved the focus from three different systems -- Windows, Windows Mobile and Xbox -- to a single system. The company moved most of the testing to virtual machines and this meant according to Berg that tests were no longer conducted on real and diverse hardware configurations for the most part.

 

Microsoft employees could self-host Windows which would mean that their machines would also be used for testing purposes. The main idea behind that was to get feedback from Microsoft employees when they encountered issues that they encountered during work days. Berg notes that self-hosting is not as widely used anymore as it was before.

 

The main source of data, apart from the automated test systems that are in place, come from Telemetry and Windows Insiders. Windows Insider builds are installed on millions of devices and Microsoft collects Telemetry from all of these devices.

 

If something crashes, Microsoft gets information about it. One of the issues associated with the collecting of Telemetry is that most bugs are not caught by it. If something does not work right, Microsoft may not be able to discern the relevant bits from Telemetry data. While it is in theory possible that users report issues, many don't and at other times, issues may go under because of other feedback that Microsoft gets from Insiders. Additionally, while Insiders may report bugs, it is often the case that necessary information is not supplied to Microsoft which poses huge issues for the engineers tasked with resolving these issues.

 

In 2014/2015, Microsoft's Testing team would be tasked with analysing bugs and issues, and supplying engineers with the data they required to resolve these.

 

Berg said engineers look at telemetry to figure out how to fix these issues and fixes are then pushed to customer devices running Insider Builds again to see if the issue got fixed or if it created new bugs.

 

To avoid total disasters like the Windows 10 version 1809 launch, gradual rollouts were introduced that would prevent feature updates from being delivered via Windows Update to the majority of machines in the early days of the release.


Closing Words

 

Microsoft exchanged the in-house Testing team with Telemetry data that it gathers from Insider Builds that it pushes to consumer and business devices, and replaced much of the PCs that it used for testing with virtual environments.

 

All of that led to an increased number of issues and bugs that customers face on production machines when installing Windows updates or feature updates.

 

Source

 

Link to comment
Share on other sites


On 9/23/2019 at 1:23 AM, Edward Raja said:

 

Wondering why in the first place Microsoft fired the testers. Their explanation on firing makes no sense at all.

they cut corners and save money on 150 less employees.

 

the user private or business needs to adress local problems

with windows. not a cost of microsoft, they might even get

hired to solve the problem they caused.

 

as a poster here mentioned if you do basic things probably

any linux version will do a better job. no forced upgrades

and you get to pick what to upgrade.

 

as for regular surfers linux firefox is as fast as it was years

ago on windows.

 

mind you if you try linux begin with usb or cd booting and

make sure wireless and audio works prior to installing it.

looking for bluetooth driver myself even with the named bt

chip there is nothing googleable.

 

Link to comment
Share on other sites


1 hour ago, halvgris said:

they cut corners and save money on 150 less employees.

 

the user private or business needs to adress local problems

with windows. not a cost of microsoft, they might even get

hired to solve the problem they caused.

 

as a poster here mentioned if you do basic things probably

any linux version will do a better job. no forced upgrades

and you get to pick what to upgrade.

 

as for regular surfers linux firefox is as fast as it was years

ago on windows.

 

mind you if you try linux begin with usb or cd booting and

make sure wireless and audio works prior to installing it.

looking for bluetooth driver myself even with the named bt

chip there is nothing googleable.

 

but when it comes to security fixes, it's mission critical.

Link to comment
Share on other sites


Has the number of bugs in Windows updates increased in the past couple of years? If so, what is the reason for the increase in bugs? That's the question that former Microsoft Senior SDET Jerry Berg, better known as Barnacules, answered in a recent video.

Berg worked for 15 years at Microsoft and one of his roles was to design and develop tools and processes to automate testing for the Microsoft Windows operating system. He left the company after Windows 8.1 shipped to the public.

Microsoft changed testing processes significantly in the past couple of years. Berg describes how testing was done in the late 2014 early 2015 period and how Microsoft's testing processes changed since then.

Back in 2014/2015, Microsoft employed an entire team that was dedicated to testing the operating system, builds, updates, drivers, and other code. The team consisted of multiple groups that would run tests and discuss bugs and issues in daily meetings. Tests were conducted manually by the team and through automated testing, and if tests were passed, would give the okay to integrate the code into Windows.

 

windows 10 display telemetry data

 

The teams ran the tests on "real" hardware in a lab through automated testing. The machines had different hardware components, e.g. processors, hard drives, video and sound cards, and other components to cover a wide range of system configurations, and this meant that bugs that affected only certain hardware components or configurations were detected in the process.

Microsoft laid off almost the entire Windows Test team as it moved the focus from three different systems -- Windows, Windows Mobile and Xbox -- to a single system. The company moved most of the testing to virtual machines and this meant according to Berg that tests were no longer conducted on real and diverse hardware configurations for the most part.

Microsoft employees could self-host Windows which would mean that their machines would also be used for testing purposes. The main idea behind that was to get feedback from Microsoft employees when they encountered issues that they encountered during work days. Berg notes that self-hosting is not as widely used anymore as it was before.

The main sources of testing data, apart from the automated test systems that are in place, comes from Telemetry and Windows Insiders. Windows Insider builds are installed on millions of devices and Microsoft collects Telemetry from all of these devices.

If something crashes, Microsoft gets information about it. One of the issues associated with the collecting of Telemetry is that most bugs are not caught by it. If something does not work right, Microsoft may not be able to discern the relevant bits from Telemetry data. While it is in theory possible that users report issues, many don't and at other times, issues may go under because of other feedback that Microsoft gets from Insiders. Additionally, while Insiders may report bugs, it is often the case that necessary information is not supplied to Microsoft which poses huge issues for the engineers tasked with resolving these issues.

Tip: you can view the Telemetry data that Microsoft collects. Also useful: find out how to configure Telemetry on Windows 10.

Back in 2014/2015, Microsoft's Testing team would be tasked with analyzing bugs and issues, and supplying engineers with the data they required to resolve these. Nowadays, Berg notes, it is Telemetry that the engineers look at to figure out how to fix these issues and fixes are then pushed to customer devices running Insider Builds again to see if the issue got fixed or if it created new bugs.

One of the main reasons why Microsoft stopped pushing out new feature updates to everyone at once was that issues that were not detected by the processed could potentially affect a large number of customers.

To avoid total disasters like the Windows 10 version 1809 launch, gradual rollouts were introduced that would prevent feature updates from being delivered via Windows Update to the majority of machines in the early days of the release.

Closing Words

Microsoft exchanged the in-house Testing team with Telemetry data that it gathers from Insider Builds that it pushes to consumer and business devices, and replaced much of the PCs that it used for testing with virtual environments.

All of that led to an increased number of issues and bugs that customers face on production machines when installing Windows updates or feature updates.  (via WinFuture)

 

Source:

https://www.ghacks.net/2019/09/23/former-microsoft-employee-explains-why-bugs-in-windows-updates-increased/

Link to comment
Share on other sites


Similar topics merged.

Link to comment
Share on other sites


Archived

This topic is now archived and is closed to further replies.

  • Recently Browsing   0 members

    • No registered users viewing this page.
×
×
  • Create New...