Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > Risk Management
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions

DW Simpson International Actuarial Jobs
Canada  Asia  Australia  Bermuda  Latin America  Europe


Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 01-17-2018, 10:05 AM
campbell's Avatar
campbell campbell is offline
Mary Pat Campbell
SOA AAA
 
Join Date: Nov 2003
Location: NY
Studying for duolingo and coursera
Favorite beer: Murphy's Irish Stout
Posts: 83,292
Blog Entries: 6
Default Operational Risk: Hawaii Missile Warning

Serious:

https://strategypage.com/on_point/20180116211630.aspx

Quote:
Learning From Hawaii's False Missile Attack Fiasco

by Austin Bay
January 16, 2018
The false missile attack alert that the Hawaii Emergency Management Agency issued at 8:05 a.m. on January 13 provides American civil defense agencies and military planners with numerous points to ponder.

Three deserve immediate attention.

Point 1: Hawaii's civil defense warning system procedures are abysmally bad.

Yes, the Wireless Emergency Alert system worked. Mobile phones instantly received over 1 million text messages.

......
This leads to an even more complex defense issue. Point 3: the incident demonstrates computer and digital device vulnerability to cyber warfare attacks, especially computers linked to alert networks.

The Hollywood movie scene has someone pushing a red button on a control console. HawaiiNewsNow.com reports that a rather mundane act generated Hawaii's false alert: a single HI-EMA employee selected the wrong option on a confusing drop-down computer menu checklist.

That menu is a mess. On the menu, the "drill" is in close proximity with other choices. Moreover, the state civil defense drill and the for real U.S. Pacific Command civil defense warning for Hawaii option (an attack warning) nest among other options, including high surf and tsunami warnings.

Tsunamis and enemy missile attacks are deadly threats. The menu's miserable clutter reflects sloppy institutional planning.

http://www.hawaiinewsnow.com/story/3...-missile-alert



Quote:
HONOLULU (HawaiiNewsNow) -
On Monday night, the governor's office released an image of what it said was the interface used to send a false missile alert to Hawaii smart phones.

But a day later, the state's emergency management agency is disputing that image and has released a different one.

Officials from the Hawaii Emergency Management Agency said the first image was sent in error, but declined to provide an actual photo of the interface.
__________________
It's STUMP

LinkedIn Profile
Reply With Quote
  #2  
Old 01-17-2018, 10:06 AM
campbell's Avatar
campbell campbell is offline
Mary Pat Campbell
SOA AAA
 
Join Date: Nov 2003
Location: NY
Studying for duolingo and coursera
Favorite beer: Murphy's Irish Stout
Posts: 83,292
Blog Entries: 6
Default

not-so-serious:

https://www.reddit.com/r/ProgrammerH...system_actual/

__________________
It's STUMP

LinkedIn Profile
Reply With Quote
  #3  
Old 01-17-2018, 11:30 AM
campbell's Avatar
campbell campbell is offline
Mary Pat Campbell
SOA AAA
 
Join Date: Nov 2003
Location: NY
Studying for duolingo and coursera
Favorite beer: Murphy's Irish Stout
Posts: 83,292
Blog Entries: 6
Default

I don't know how real the following is:

http://www.businessinsider.com/hawai...iticism-2018-1

Quote:
A password for the Hawaii emergency agency was hiding in a public photo, written on a Post-it note

A false alert warning of an inbound missile was broadcast in Hawaii on Saturday.
Since then, people have discovered that a photo taken in Hawaii's Emergency Management Agency for a news article in July includes a sticky note with a password.
Hawaii says the alert was sent was because "an employee pushed the wrong button," not because of a hack, but the photo has sparked criticism about the agency's level of security.

On Saturday, people in Hawaii were awakened by a terrifying false alert about an inbound missile. Hawaii's Emergency Management Agency has said a worker clicked the wrong item in a drop-down menu and sent it, and that its system was not hacked.

"It was a mistake made during a standard procedure at the changeover of a shift, and an employee pushed the wrong button," Gov. David Ige said.

But an Associated Press photo from July that recently resurfaced on Twitter has raised questions about the agency's cybersecurity practices.

In it, the agency's operations officer poses in front of a battery of screens. Attached to one is a password written on a Post-it note.

Spoiler:





Quote:
An agency spokesman told Hawaii News Now that the password is authentic, and had been used for an "internal application" that he believed was no longer being used.

While these computers are unrelated to the system that sent the false missile alert, the photo raises questions about the approach to information security at the agency. (On the other screen, another note reminds the user to "SIGN OUT.")

Writing down passwords isn't a strict security no-no. Some experts say that keeping a hard copy of a password in your wallet is defensible — if you can keep the piece of paper secure. But a note on a monitor is not secure, especially if it's for computer systems dedicated to keeping people safe.

The photo has already drawn some ridicule from those in the operational-security industry.

__________________
It's STUMP

LinkedIn Profile
Reply With Quote
  #4  
Old 01-18-2018, 09:10 AM
ronaldy27's Avatar
ronaldy27 ronaldy27 is offline
Member
CAS
 
Join Date: Jan 2012
Location: The South
Studying for the credential
Posts: 2,972
Default

Quote:
Originally Posted by campbell View Post
haha I love that
__________________
Spoiler - for size:



Reply With Quote
  #5  
Old 01-18-2018, 09:53 AM
campbell's Avatar
campbell campbell is offline
Mary Pat Campbell
SOA AAA
 
Join Date: Nov 2003
Location: NY
Studying for duolingo and coursera
Favorite beer: Murphy's Irish Stout
Posts: 83,292
Blog Entries: 6
Default

https://medium.com/ux-immersion-inte...s-d30d59ddfcf5

Quote:
The Hawaii Missile Alert Culprit: Poorly Chosen File Names
Spoiler:
Saturday morning, January 13, 2018 at 8:09am Hawaii time, a staff member of the Hawaii Emergency Management Agency’s (HIEMA) State Warning Point office was going through their routine shift change checklist. They went through the same checklist every time they started their shift. It was routine. It wasn’t interesting.

At one point, they opened up their IPAWS alert software, retrieved a list of saved “templates” and picked one from a list of 9. What they picked was named PACOM (CDW) — STATE ONLY.

Only, this wasn’t the template file they meant to open. The template they meant to open was named DRILL — PACOM (CDW) — STATE ONLY. Other than the word DRILL in the file name, the two files were nearly identical. I say nearly, because there was one other difference: The drill version sent a message only to test devices, while the non-drill version sent the exact same message to every mobile phone in Hawaii.

The message was ominous. BALLISTIC MISSILE THREAT INBOUND TO HAWAII. SEEK IMMEDIATE SHELTER. THIS IS NOT A DRILL.

The State Warning Point staff member didn’t immediately realize they’d chosen the wrong file. Nothing in the system would tell them they had. The clicks and confirmations were exactly the same for either file. They wouldn’t realize their mistake until a few moments later, when their personal phone buzzed with the alert, as did everyone else’s in the State Warning Point Emergency Operations Center. Oops.

Sending a message to millions of phones about an incoming ballistic missile should, one would think, have a confirmation message. It did. But so did the test message. It also required the user type in a special password to ensure they intended to send the message to every recipient, but so did the test message.

The crux of the error was choosing the wrong file, from a list that looked like this:

[img]https://cdn-images-1.medium.com/max/800/1*g8apgO_BtMFaH_RC-W5jPw.jpeg[/img]


This list, supplied by HIEMA, are the templates for messages.
It’s not hard to see how the wrong file was chosen. These seem to be listed in chronological order, newest to last. (The first file BMD False Alarm was added right after the false alert went out and the Governor issued a statement. It contains the message used to tell everyone that the NOT A DRILL message in fact was a DRILL.)

There’s an alternate template list floating around: (It seems the state isn’t sure which State Warning Point uses.)
[img]https://cdn-images-1.medium.com/max/800/1*jQhE_C6YVZaObBZDap1uaw.jpeg[/img]

In either case, you clearly see that the DRILL file name is almost identical to the actual file name. (CDW is Civil Defense Warning.) This list is alphabetical, which isn’t much better than date ordered.

Planning For This Mistake To Happen
This wasn’t a thoughtfully designed menu, any more than any random collection of user-named objects are. Yet, to get here, a lot of careful planning had to happen.

The system used to send out the alerts is called IPAWS — the Integrated Public Alert & Warning System — managed by the US Government’s Federal Emergency Management Agency (FEMA) in association with the Federal Communications Commission (FCC). The cellphone portion of IPAWS is known as the Wireless Emergency Alert (WEA) system.

States and counties can get access to the IPAWS system to post emergency notices, from everything from road closures to AMBER alerts (child abduction alerts).

Hawaii has had their system in place for a while, which they used for weather related issues, like landslides that close roads and tsunami warnings. In November, HIEMA concerned about rising tensions with North Korea, put out a new emergency preparedness plan, which included WEA messaging in case of a ballistic missile launch.

Here’s a slide from HIEMA’s emergency preparedness slide deck, stating their reaction to a missile launch:

[img]https://cdn-images-1.medium.com/max/800/1*yb32EYh7SXuFkKbrcfBM_Q.png[/img]

Here are the messages HEIMA recommended:

[img]https://cdn-images-1.medium.com/max/800/1*AnhFZttgCzp7z57NjMSd6w.png[/img]


You can see the actual message that went out was very close to the messages recommended in HIEMA’s plan. (WEA has a 90-character limit, so the recommended 138-character message needed to be shortened.)

I’m guessing, with these new guidelines in place, the State Warning Point updated their saved messages, first including the drill as a regular practice, then adding the actual message. (It seems they decided to add a template for tsunami warnings at the same point, as that fall in between to the two PACOM messages. PACOM stands for the United States Pacific Command, the military joint coordination office that monitors the Pacific region.)

FEMA lists 23 approved vendors of IPAWS alert origination software. Most of these are either Software as a Service or iOS apps. (Yes, someone can alert their entire state from your phone.)

HIEMA hasn’t revealed which vendor they’re currently using. (I’ve been told FOIA requests have already been filed to learn more about the systems used in Hawaii and other states.) It’s unlikely the HIEMA implementation was customized in any way. It takes quite a bit of work to get FEMA certified. Leave that work to the vendors.

If you visit the websites for the various approved vendors, you see some nicely designed systems. Here’s one from a company called Alertsense:

[not embedding any more of these, go to the original link]

It seems to be a cleanly designed system. However, it has a predecessor and it’s possible HIEMA is using something that looks more like this:

[image at the link]

Governments, which are often about saving their taxpayers money, don’t pay for upgrades. Especially for systems that aren’t broken. Hopefully, we’ll learn more soon about the specific system Hawaii is using.

This Mistake Has Likely Happened Before
My reaction, upon seeing the system involved, was why hasn’t this happened before? After all, choosing a wrong file is a common mistake. If the way you send an IPAWS message is to select a predefined template, then it’s likely someone picked the wrong one in the past.

However, it wouldn’t make the news. Most IPAWS alerts are for AMBER alerts and local weather issues. If the wrong message was sent out, say for a road closed by flooding, most people wouldn’t know there was an issue. If they weren’t near that particular road, they probably wouldn’t give attention to the message at all, even if it was a mistake.

IPAWS messages are usually local. A statewide message is rare. That’s what made this incident so public.

The newsworthiness of the incident was amplified because of our nation’s recent tensions with North Korea. It seems we’re always just a very stable genius’s tweet away from having a missile launched in our direction. Had this accident happened two years ago, everyone would’ve had a different reaction. (Though, it wouldn’t have happened two years ago because there was no need to think about incoming ballistic missiles back then.)

File Names Are The Culprit
Have you every known someone who loaded an old version of a file, when they meant to use the most recent, then accidentally saved the old data over the new? That is, in essence, the problem we saw here.

This is a classic user experience problem involving file name conventions. Users often do not choose file names thinking will I make a mistake in the future and choose the wrong file? We make mistakes with our files all the time, grabbing the wrong version when two files are similarly named.

If we want to ask ourselves, how would we prevent this from happening again, we have to look at how the IPAWS messages are stored for future use. The system, as it works today, relies on users to save the message templates with a meaningful name. If they fail to do so. If they choose a name that’s quite similar to another, this mistake will happen again.

There’s no system enforcing naming convention, just like there aren’t naming conventions for any other files in any other filesystem. Our systems don’t force a specific file naming regimen on the users, to ensure that every name is clear and distinct, so they’ll prevent naming confusion in the future.

A Possible Solution — Get Rid of File Names
In many ways, file names are an anachronism from days of old, when reading data was slow and space was costly. We could eliminate them in many cases, and this is one of them.

For an IPAWS WEA message, there are two distinct identifiers that we could use instead of a file name: the message itself and who it’s broadcast to. The message (BALLISTIC MISSLE THREAT INBOUND) is the most important thing and would distinguish it from other messages. The design could use the message as a primary indicator.

The design could also separate drills from the actual emergency alerts. There could be separate lists for the drills, which are clearly distinguished by location and other distinct visual coding (like color shading).

Using the message as the identifier and separating out the drills would do a lot to prevent this problem from re-occurring. Frankly, this is a classic micro-interactions problem. Good design could go a long way to make sure we never have this problem again.

Of course, this would be up to the vendor of the software. And there’s 23 vendors in this case. Is it FEMA’s responsibility to mandate this type of UX change? Or would the vendors do it just to avoid embarrassment?

We probably don’t want our lawmakers creating safety standards for file name methodologies. Or do we?

How Would We Have Predicted This Problem?
It’s easy now, with 20–20 hindsight, to see this problem clearly. But could we have predicted it the week before?

We don’t do enough work to stress test our own designs. We don’t ask whether people will confuse names they created themselves because those names are too similar. We don’t ask how we prevent panicking the population of an entire state, clogging up emergency 911 call centers, possibly putting anyone having a real emergency at risk.

In emergency preparedness operations, drills are a regular practice. FEMA, state emergency operations, and local first responders regularly hold practice drills. In these drills, they try to anticipate what might go wrong. They create scenarios that push at the extremes (what if a terrorist attack happened during hurricane recovery?), then hold retrospectives to take apart what happened and where the system didn’t hold up.

In these drills, product and service vendors are invited to participate. They observe how their products and services held up under the stress of real situations. Smart vendors also run their own drills and simulations, pushing their own products and services to extremes. These stress test their products, giving them a chance to make them more robust.

These types of procedures are not common in digital products, except in the information security space, where regular hacking events have become common. (Last year, the Department of Defense U.S. Digital Service invited the public to break into D.O.D. systems, offering a bounty for anyone who could succeed, to stress their system security.)

What would a simulated missile launch have revealed about the operation of the system? Would we have encountered the exact opposite of the problem that happened on January 13? Would we see an operator using the DRILL template when they meant to use the official warning? In HIEMA’s emergency action plan, the WEA message needs to go out 5 minutes after detection because impact happens 15 minutes later. How many lives might be lost if a delay happened because it took 5 minutes to realize no warning went out?

We need more of these types of learning experiences, where vendors and operators work to stress our systems. We need to break down the barriers of organizations and work directly with the users, as they have a lot of perspective to offer us. Then we need to design something better, something safer for that population.
__________________
It's STUMP

LinkedIn Profile
Reply With Quote
  #6  
Old 01-18-2018, 09:53 AM
campbell's Avatar
campbell campbell is offline
Mary Pat Campbell
SOA AAA
 
Join Date: Nov 2003
Location: NY
Studying for duolingo and coursera
Favorite beer: Murphy's Irish Stout
Posts: 83,292
Blog Entries: 6
Default

And yeah, I know why the images aren't embedding, and no, I'm not going to fix it.

Stupid file names.
__________________
It's STUMP

LinkedIn Profile
Reply With Quote
  #7  
Old 01-18-2018, 02:24 PM
nonlnear nonlnear is offline
Member
Non-Actuary
 
Join Date: May 2010
Favorite beer: Civil Society Fresh IPA
Posts: 29,627
Default

Quote:
Originally Posted by campbell View Post
Wow, that article author totally misses the point. The problem isn't the bad file names, it is that the interface design process was fundamentally flawed in that it clearly failed to deliberately design for quality.
Reply With Quote
  #8  
Old 01-18-2018, 02:25 PM
wat?'s Avatar
wat? wat? is offline
Member
SOA AAA
 
Join Date: May 2004
Location: Hi
Studying for beer
Favorite beer: Fresh Squeezed IPA
Posts: 33,136
Default

Quote:
Originally Posted by campbell View Post


Best part is the "oh shit" pointer shake
__________________
"Mathematical Induction: How mathematicians manage to suck all the fun out of lining up a row of dominos, knocking the one on the end down, and watching the entire row fall." -BC
Skip it. - AG

AO Golfers Unite! Here and here.
Chase Sapphire Reserve Referral? clicky
Reply With Quote
  #9  
Old 01-18-2018, 02:30 PM
wat?'s Avatar
wat? wat? is offline
Member
SOA AAA
 
Join Date: May 2004
Location: Hi
Studying for beer
Favorite beer: Fresh Squeezed IPA
Posts: 33,136
Default

If we're sending out emergency messages all at once by clicking individual links, it's a problem.

Everyone's talking about a "big red button". Well, if they are indeed emergency situations, then let's go ahead and make them "big red buttons". If it's a test, then make them yellow. If it's a maintenance message/regular update, make it a green button.

Part of operational risk, I think, can be alleviated by understanding the human psyche and constructing your processes around what comes naturally to people. Most adults will get the concept behind a green/yellow/red color schematic.
__________________
"Mathematical Induction: How mathematicians manage to suck all the fun out of lining up a row of dominos, knocking the one on the end down, and watching the entire row fall." -BC
Skip it. - AG

AO Golfers Unite! Here and here.
Chase Sapphire Reserve Referral? clicky
Reply With Quote
  #10  
Old 01-20-2018, 06:00 AM
bitter buffalo's Avatar
bitter buffalo bitter buffalo is offline
Member
SOA
 
Join Date: Aug 2010
Favorite beer: trappistes rochefort 10
Posts: 1,008
Default

__________________
It may be the coldest day of
the year, what does he think of
that? I mean, what do I? And if I do,
perhaps I am myself again.
Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 07:39 PM.


Powered by vBulletin®
Copyright ©2000 - 2018, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.54946 seconds with 9 queries