Baltimore ransomware nightmare could last weeks more, with big consequences

[Karlston]

Houses can't be sold, bills can't be paid while city networks are shuttered.

Enlarge / Days after Mayor "Jack" Young took over for disgraced Baltimore Mayor Catherine Pugh, ransomware took down Baltimore City's networks. It may be weeks or months before things return to normal—and "normal" wasn't that great, either, based on the city's IT track record.

Alex Wroblewski/Getty Images

It's been nearly two weeks since the City of Baltimore's networks were shut down in response to a ransomware attack, and there's still no end in sight to the attack's impact. It may be weeks more before the city's services return to something resembling normal—manual workarounds are being put in place to handle some services now, but the city's water billing and other payment systems remain offline, as well as most of the city's email and much of the government's phone systems.

 

The ransomware attack came in the midst of a major transition at City Hall. Mayor Bernard C. “Jack” Young assumed office officially just days before the attack, after the resignation of former mayor Catherine Pugh, who is facing an ever-expanding corruption investigation. And some of the mayor's critical staff positions remained unfilled—the mayor's deputy chief of staff for operations, Sheryl Goldstein, starts work today.

 

To top it off, unlike the City of Atlanta—which suffered from a Samsam ransomware attack in March of 2018—Baltimore has no insurance to cover the cost of a cyber attack. So the cost of cleaning up the RobbinHood ransomware, which will far exceed the approximately $70,000 the ransomware operators demanded, will be borne entirely by Baltimore's citizens.

 

It's not like the city wasn't warned. Baltimore's information security manager warned of the need for such a policy during budget hearings last year. But the final budget did not include funds for that policy, nor did it include funding for expanded security training for city employees, or other strategic investments that were part of the mayor's strategic plan for the city's information technology infrastructure.

This may take a while

In a statement to press on May 17, Mayor Young said:

I am not able to provide you with an exact timeline on when all systems will be restored. Like any large enterprise, we have thousands of systems and applications. Our focus is getting critical services back online, and doing so in a manner that ensures we keep security as one of our top priorities throughout this process. You may see partial services beginning to restore within a matter of weeks, while some of our more intricate systems may take months in the recovery process… we engaged leading industry cybersecurity experts who are on-site 24-7 working with us.

Some of the restoration efforts also require that we rebuild certain systems to make sure that when we restore business functions, we are doing so in a secure manner.

City officials have provided few details about the extent of the attack, as the city is cooperating with an FBI investigation. But it appears that the ransomware was triggered on some systems in the early hours of May 7, when email service was suddenly interrupted. The city's response to the attack has thrown many city services into disorder or shut them down entirely.

 

The attack was first reported by Baltimore's Department of Public Works, when the department's official Twitter account announced that its email access was cut off, and it reported phones and other systems were affected soon afterward. As it became clear what was happening, the city's Office of Information Technology team shut down nearly all of the city's non-emergency systems to prevent the further spread of the attack. It’s not clear how widespread the ransomware was within the network, but the city's email and IP-based phones were among the systems affected.

 

City officials have stressed that emergency systems, such as police and fire department networks and the city's 911 system, were not affected. The 911 system suffered from a ransomware attack last year when some firewall settings were disabled during maintenance. But the Baltimore Police Department was dependent on the city's email servers, and surveillance cameras around the city have been affected by the network shutdown. Nearly every other city department had services interrupted as well.

 

Real estate purchases cannot be closed, though Mayor Young said that a paper-based workaround for handling closings would be put in place by today. Water bills and other city charges (including parking tickets and citations from the city's speed camera and red light camera network) cannot be paid. And many city workers have had to resort to using their own laptops without a connection to city networks, as well as personal e-mail addresses and cell phones, in order to get work done. Other tasks are idled completely or have gone back to paper-based processes the city was in the midst of trying to eliminate.

A thankless job

The mayor's Office of Information Technology has been struggling to regain its footing over the past two years after a string of fired chief information officers—four consecutive CIOs were fired or forced to resign over a period of five years. Frank Johnson, who now holds the titles of both CIO and Chief Digital Officer for the city, was hired in November 2017 after leaving a position as a regional vice president of sales for Intel. Johnson led the development of a digital strategy for the city that aimed to bring Baltimore's IT spending more in line with those of similarly sized cities and transform its IT practices. According to a 2018 strategy document, Baltimore spends about half of what other cities budget for IT, and the Office of Information Technology only controls about one percent of the total budget; most of the IT spending is part of other department's operational budgets.

 

Until the ransomware attack, the city's email was almost entirely internally hosted, running on Windows Server 2012 in the city's data center. Only the city's Law Department had moved over to a cloud-based mail platform. Now, the city's email gateway has moved to a Microsoft-hosted mail service, but it's not clear whether all email will be migrated to the cloud—or if it's even possible. While Mayor Young said the city had data backups, it's not clear how widely backups were implemented. And Johnson would not say whether there was a disaster-recovery plan in place to deal with a ransomware attack.

 

Some of Baltimore's systems are hosted elsewhere, including the city's primary website, which is hosted on Amazon Web Services and operated by a contractor. But the city almost lost that website last week, and not because of ransomware: the contract for operating the site had expired, and the city was delinquent in its payments.

 

Tracking down how and when the malware got into the city's network is a significant task. The city has a huge attack surface, with 113 subdomains—about a quarter of which are internally hosted—and at least 256 public IP addresses (of which only eight are currently online, thanks to the network shutdown).

 

"We engaged leading industry cybersecurity experts who are on-site 24-7 working with us," Young said. "As part of our containment strategy, we deployed enhanced monitoring tools throughout our network to gain additional visibility. As you can imagine, with approximately 7,000 users, this takes time."

 

Source: Baltimore ransomware nightmare could last weeks more, with big consequences (Ars Technica)

[coromonadalix]

maybe this time they'll learn ??

[Karlston]

Somebody lend Baltimore $6 —

Google bots shut down Baltimore officials’ ransomware-workaround Gmail accounts

Google automatically suspended accounts after detecting they were from same network.

Enlarge / Oh, Baltimore.

Alex Wroblewski/Getty Images

In the wake of the ransomware attack that has kept city networks and infrastructure shut down now for over two weeks, Baltimore officials—including the mayor and city council members—set up Google Gmail accounts as a backup communications channel. But earlier this week, Google's automated systems shut the accounts down, instructing the account holders to purchase a business account.

 

On May 23, a Google spokesperson said through the company's Twitter account, "We have restored access to the Gmail accounts for the Baltimore City officials. Our automated security systems disabled the accounts due to the bulk creation of multiple consumer Gmail accounts from the same network."

The problem could have been prevented if Baltimore City officials had set up a Google GSuite Government account (or even just a regular GSuite account) at $6 per user per month.

 

Baltimore Mayor Bernard "Jack" Young and other city leaders were not available for comment about the issue, as they were in Las Vegas attending the International Council of Shopping Centers’ convention this week. At the convention, Mayor Young told reporters his agenda was to tell developers that "Baltimore is open for business."

Well, maybe not so open

Enlarge / A screenshot of Baltimore's website. Email and phone services are still not working for most city employees, and some services are functioning through manual workarounds.

Many city employees are making do during the recent system outages by using personal email accounts and cell phones to conduct city business. The city's leadership has also devised a number of "manual workarounds" to restore some government functions—including the city's ability to process real estate transactions.

 

Even though the city's financial records are on a mainframe unaffected by the ransomware, the city has shut down all its internal networks to prevent any further spread of malware—making it impossible for employees to create the lien certificates required to process deeds, for instance. For now, the city requires all transactions to be completed in person from 7am to 7pm, and sellers are required to sign an affidavit that they will pay any outstanding taxes or other liens on the property within 10 days of being invoiced by the city (whenever that may be). The lien certificates then have to be hand-delivered by the seller to another office.

 

"This solution to the pause in real estate transactions essentially removes any risk stemming from existing liens from the new owner of the property, allowing title insurance companies to continue with their normal course of business in Baltimore," a Baltimore spokesman said in a release issued on May 21.

 

Source: Google bots shut down Baltimore officials’ ransomware-workaround Gmail accounts (Ars Technica)

[lolsee2]

They should do a bounty to have tech savvy individuals discover weaknesses in systems and infrastructure so that they can patch those up before they are exploited for nefarious purposes.

 

Sort of like how Apple does with their tech products and Nintendo with their consoles.