Difference between revisions of "ADVISE Meta Alpha Tutorial"

Return to Workshop Home

This page is home for the tutorial demonstrating the ADVISE Meta Alpha Tool.

Background[edit]

The goal of this tutorial is to demonstrate the ADVISE Meta Tool. We create two models of an electric distribution system, one with firewalls with weak administrator authentication and one with firewalls with strong administrator authentication, and compare their vulnerability in the face of an attack by an insider. The quantitative security evaluation enabled by this tool may help a system architect select the appropriate design for their security posture.

In this tutorial we will build the model from scratch. Here is a completed version of the project built using this tutorial.

Creating a Model for Security Evaluation Using the ADVISE Meta Tool[edit]

The first time you run the alpha tool, you may be prompted for your Mobius license. Enter your Mobius user account or select the license file you have downloaded.

Importing an Ontology[edit]

Before we can start creating ADVISE Meta models, we need an ontology to work with. Download the ontology used for this tutorial.

The first time you open Mobius, the Ontology view may not be visible. Show it by selecting the menu Window -> Show View -> Ontology Explorer.

To import the ontology

1. Right click on the white space in the Ontology view.
2. Select Import...
3. Navigate to the ontology you downloaded previously, select it and press Open.

Creating an ADVISE Meta Model[edit]

Creating a new project[edit]

1. Switch from the Ontology view to the Projects view by click on the Projects tab in the upper left corner of the window.
2. Right click the Open Projects folder and select New Project.
3. Name the project AdviseMetaTutorial.

Creating global variables[edit]

We need to create two global variables for this tutorial.

1. Right click on the AdviseMetaTutorial project folder, and click Properties.
2. Double click the Global Variables item in the list.
3. Add a global variable of type character called userAuthType.
   1. Click Add.
   2. Enter userAuthType for the name.
   3. Select Character from the Type drop down menu.
   4. Click Finish.
4. Add a global variable of type short called userAuthStrength
   1. Click Add.
   2. Enter userAuthStrength for the name.
   3. Leave Short selected in the Type drop down menu.
   4. Click Finish.
5. Click OK to exit.

Creating a New ADVISE Meta Atomic Model[edit]

1. Right click on the AdviseMetaTutorial project folder and select New.
2. Select Atomic in the bottom pane and click Next.
3. Select the ADVISE Meta Model from the list, enter the name MetaModel, and click Finish.

Creating a System Model[edit]

The system model represents the components of the system of interest and the relationships among these components. The components may include (but are not limited to) networks, firewalls, and workstations, for example. The onNetwork relationship is an example of a relationship that may exist between a workstation and a network.

Creating Networks[edit]

1. In the components tree on the left, expand the PhysicalThing component node to make the Device and Network component types visible.
2. Add a Network component called EngrLAN
   1. Drag and drop a Network component on to the diagram.
   2. Click on the new Network 1 component and select the Edit Details button.
   3. Change the name to EngrLAN, but leave the other attributes set to the defaults.
   4. Click Finish.
3. In the components tree on the left, expand the Network component node to make the WiredNetwork and WirelessNetwork component types visible.
4. Add a WiredNetwork component called CorpLAN and leave the attributes set to the defaults.
5. Add a WiredNetwork component called SCADALAN and leave the attributes set to the defaults.

Creating Firewalls[edit]

1. In the components tree on the left, expand the Device component node under the PhysicalThing node to make the Firewall and Host component types visible, then expand the Firewall component node to make the FirewallAppliance and FirewallHosted component types visible.
2. Add a FirewallHosted and name it CorpLanScadaLanFW. Change strengthOfUserAuthentication from 0 to 1, and change userAuthenticationType from N to W. The variable strengthOfUserAuthentication is on a 0-10 scale. The variable N represents no authentication, and W represents weak authentication.
3. Add another FirewallHosted and name it CorpLanEngrLanFW. Change strengthOfUserAuthentication from 0 to 1, and change userAuthenticationType from N to W.
4. Add another FirewallHosted and name it EngrLanScadaLanFW. Change strengthOfUserAuthentication from 0 to 1, and change userAuthenticationType from N to W.
5. Add a new onNetwork relationship from the CorpLanScadaLanFW to the corpLAN component.
   1. Select the CorpLanScadaLanFW component, click Add Relationship, and click on the corpLAN component.
   2. Select the onNetwork relationship from the drop down list.
   3. Click Finish.
6. Add a new onNetwork relationship from the CorpLanScadaLanFW to the SCADALAN component.
7. Add a new onNetwork relationship from the EngrLanScadaLanFW to the SCADALAN component.
8. Add a new onNetwork relationship from the EngrLanScadaLanFW to the EngrLAN component.
9. Add a new onNetwork relationship from the CorpLanEngrLanFW to the EngrLAN component.
10. Add a new onNetwork relationship from the CorpLanEngrLanFW to the CorpLAN component.

Creating an SSH Server on a Linux Workstation[edit]

1. In the components tree on the left, expand the Host component node to make the FirewallHosted, Server and Workstation component types visible.
2. Add a new Workstation and name it EngrWorkstation.
3. Add a new onNetwork relationship from the EngrWorkstation to the EngrLAN component.
4. In the components tree on the left, expand the System component node to make the Device and Software component types visible, then expand the Software component node to make Application and OperatingSystem visible.
5. Add a new OperatingSystem and name it LinuxOS.
6. Add a new hardwarePlatform relationship from the LinuxOS to the EngrWorkstation, in the same way you created the onNetwork relationships.
7. Add a new Application and name it SSHServer.
8. Add a new applicationOS relationship from the SSHServer to the LinuxOS, in the same way you created the onNetwork and hardwarePlatform relationships.

Creating an HMI on the SCADALAN[edit]

1. Add a new Host and name it HMI.
2. Add a new onNetwork relationship from the HMI to the SCADALAN component.

Finish the System Model[edit]

1. Click the File -> Save menu item.

This is an example of the complete ADVISE Meta system diagram.

Defining the Goals[edit]

1. Select the Goals tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Select the New Goal 1 goal from the list in the upper right.
3. Change the name below to read Goal_GainNetworkAccessOnScadaNetwork
4. Add the SCADALAN_NetworkAccess state variable to the list of state variables
   1. Expand the SCADALAN component in the Available State Variables tree on the left side of the window.
   2. Click and drag the NetworkAccess state variable, and drop it in the list of Dependent State Variables.
5. Change the text in the Goal Expression to the following code:

   return SCADALAN_NetworkAccess->Mark();
   

6. Click the File -> Save menu item.

Creating an Adversary Model[edit]

1. Select the Adversaries tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Click and drag the IndependentInsider adversary template over to the Adversaries list on the right side.
3. Change the name below to EngineerInsider.
4. In the Access section, add the access elements the adversary holds at the beginning of the attack.
   1. Click the Add button.
   2. Select the CorpLAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   3. Select the CorpLanEngrLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   4. Select the CorpLanScadaLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   5. Select the EngrLAN_NetworkAccess element in the left pane, and click the > button to move it to the right pane.
   6. Select the EngrLAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   7. Select the EngrLanScadaLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   8. Select the EngrWorkstation_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
   9. Select the EngrWorkstation_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
   10. Select the EngrWorkstation_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   11. Select the EngrWorkstation_UIAccess element in the left pane, and click the > button to move it to the right pane.
   12. Select the HMI_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   13. Select the LinuxOS_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
   14. Select the LinuxOS_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
   15. Select the LinuxOS_UIAccess element in the left pane, and click the > button to move it to the right pane.
   16. Select the SCADALAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   17. Select the SSHServer_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
   18. Select the SSHServer_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
   19. Select the SSHServer_UIAccess element in the left pane, and click the > button to move it to the right pane.
   20. Click Finish.
5. The adversary does not start the attack with any special knowledge of the system or customizable skills, so do not modify the adversary's knowledge or skill elements.
6. In the Goals section, add the Goal_GainNetworkAccessOnScadaNetwork goal.
   1. Click the Add button.
   2. Select the Goal_GainNetworkAccessOnScadaNetwork element in the left pane, and click the > button to move it to the right pane.
   3. Select the Goal_GainNetworkAccessOnScadaNetwork element, and then press the Payoff cell and enter the value 80000.
   4. Click Finish.
7. Click the File -> Save menu item.

Creating a Configuration[edit]

1. Select the Configurations tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Select the New Configuration 1 configuration from the list on the left.
3. Change the name on the right side to read WeakFirewalls
4. Add the goal defined previously to the configuration.
   1. Click the Add button in the Goals section.
   2. Select the Goal_GainNetworkAccessOnScadaNetwork goal from the list.
   3. Click Finish.
5. Select EngineerInsider as the adversary for this configuration from the drop down list.
6. TODO For now leave the metrics blank.
7. Click the File -> Save menu item.

Generating an ADVISE Model[edit]

To generate the ADVISE Model:

1. Select the Generator tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Select the WeakFirewalls configuration from the list on the left.
3. Click the Generate button.
4. Click on the File->Close menu item.

To compile the ADVISE Mode:

1. Open the newly created ADVISE model called WeakFirewalls.
2. Click on the File->Save menu button. This will automatically compile the model.
3. After compilation press the File->Close menu item.

Defining the Reward Model[edit]

We will define a performance variable that will track whether the adversary achieved the goal over a 24-hour period.

1. Expand the Reward subfolder in the projects view to reveal WeakFirewallsPVs and double click it to open it.
2. Add a new variable called GoalAchieved by entering "GoalAchieved" into the textbox in the top left of the dialog and then pressing "Add Variable:".
3. In the "Rate Rewards" tab, define the reward function to be:

   return WeakFirewalls->Goal_GainNetworkAccessOnScadaNetwork->Mark();
   

4. Click on the "Apply Changes" button.
5. In the "Time" tab, define the timing type to be an "Instant of Time" with an incremental range. The range should be 0-60 with a step size of 10.
   1. Select the Time Tab
   2. From the Type drop down list select Instant of Time.
   3. From the Time Point definition method select Incremental Range.
   4. In the First time point in series textbox enter 0.0
   5. In the Upper Bound of series textbox enter 24.0
   6. In the Step size in series textbox enter 2.0
   7. Click on the "Apply Changes" button.
6. Save and close the reward model by clicking File->Save and File->Close.

This defines a reward model for your ADVISE atomic model that measures the state of the goal over a 24-hour period.

Defining the Study[edit]

1. Compile the Range Study.
   1. Expand the Study subfolder in the Projects view and double click the study called StrongFirewallsStud
   2. Compile and close the study by clicking on the File->Save and File->Close menu items.

Running the Model[edit]

1. Expand the Solver subfolder in the Projects view and double click the WeakFirewallsSim simulator.
2. Click on the "Run Simulation" tab and click the "Start Simulation" button.
3. Wait for the simulation to complete, this may take several minutes depending on the speed of your machine. When the simulation finishes you should now be looking at the "Results" tab.
4. Scroll down to see the mean values of the performance variable at the various time points.
5. Values should be similar to

_________________________________Mean Results_________________________________
Name                  Time               Mean                     Confidence Interval
GoalAchieved          0.0               0.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          2.0               2.1000000000E-03  +/-   8.9728611192E-04  (*)
GoalAchieved          4.0               2.8200000000E-02  +/-   3.2448213338E-03  (*)
GoalAchieved          6.0               9.8300000000E-02  +/-   5.8356008775E-03
GoalAchieved          8.0               1.8260000000E-01  +/-   7.5726082533E-03
GoalAchieved          10.0              2.5560000000E-01  +/-   8.5499070658E-03
GoalAchieved          12.0              3.2230000000E-01  +/-   9.1606582755E-03
GoalAchieved          14.0              3.7860000000E-01  +/-   9.5072237807E-03
GoalAchieved          16.0              4.3560000000E-01  +/-   9.7188577416E-03
GoalAchieved          18.0              4.8330000000E-01  +/-   9.7950219940E-03
GoalAchieved          20.0              5.3120000000E-01  +/-   9.7813910489E-03
GoalAchieved          22.0              5.7350000000E-01  +/-   9.6940223358E-03
GoalAchieved          24.0              6.1140000000E-01  +/-   9.5541466358E-03

Click the File->Close menu item.

According to this model there is an approximately 61% chance that the Engineer Insider will gain unauthorized access to the SCADA network 24 hours after beginning the attack with this system configuration. We stress that quantitative metrics calculated by ADVISE models are not intended to stand alone. Rather, these metrics are supposed to be compared to metrics that come from similar models. In the next section we will build a model of a similar system with stronger firewalls.

Creating a Second Model for Security Evaluation Using the ADVISE Meta Tool[edit]

Creating another ADVISE Meta Model[edit]

Creating a New ADVISE Meta Atomic Model[edit]

1. Right click on the MetaModel1 ADVISE Meta Model created in the previous section. Select Copy and give it the name MetaModel2. Click Finish.

Modifying the Meta Model[edit]

We will use the same system configuration but with strengthened firewalls.

Strengthen Firewalls[edit]

1. Open MetaModel2 by double clicking it.
2. Select CorpLanScadaLanFW, click on Edit Details, and change strengthOfUserAuthentication from 1 to 8, and change userAuthenticationType from W to T. The variable strengthOfUserAuthentication is on a 0-10 scale. The variable W represents weak authentication, the variable T represents two-factor authentication.
3. Select CorpLanEngrLanFW, click on Edit Details, and change strengthOfUserAuthentication from 1 to 8, and change userAuthenticationType from W to T.
4. Select EngrLanScadaLanFW, click on Edit Details, and change strengthOfUserAuthentication from 1 to 8, and change userAuthenticationType from W to T.

Finish the System Model[edit]

1. Click the File -> Save menu item.

Creating a New Configuration[edit]

1. Select the Configurations tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Change the name of the configuration to StrongFirewalls
3. Click the File -> Save menu item.

Generating an ADVISE Model[edit]

To generate the ADVISE Model:

1. Select the Generator tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Select the StrongFirewalls configuration from the list on the left.
3. Click the Generate button.
4. Click on the File->Close menu item.

To compile the ADVISE Mode:

1. Open the newly created ADVISE model called StrongFirewalls.
2. Click on the File->Save menu button. This will automatically compile the model.
3. After compilation press the File->Close menu item.

Defining the Reward Model[edit]

We will define a performance variable that will track whether the adversary achieved the goal over a 24-hour period. This is exactly the same process used to create the reward variable previously described in the tutorial.

1. Expand the Reward subfolder in the projects view to reveal StrongFirewallsPVs and double click it to open it.
2. Add a new variable called GoalAchieved by entering "GoalAchieved" into the textbox in the top left of the dialog and then pressing "Add Variable:".
3. In the "Rate Rewards" tab, define the reward function to be:

   return StrongFirewalls->Goal_GainNetworkAccessOnScadaNetwork->Mark();
   

4. Click on the "Apply Changes" button.
5. In the "Time" tab, define the timing type to be an "Instant of Time" with an incremental range. The range should be 0-60 with a step size of 10.
   1. Select the Time Tab
   2. From the Type drop down list select Instant of Time.
   3. From the Time Point definition method select Incremental Range.
   4. In the First time point in series textbox enter 0.0
   5. In the Upper Bound of series textbox enter 24.0
   6. In the Step size in series textbox enter 2.0
   7. Click on the "Apply Changes" button.
6. Save and close the reward model by clicking File->Save and File->Close.

This defines a reward model for the ADVISE atomic model that measures the state of the goal over a 24-hour period.

Defining the Study[edit]

1. Compile the Range Study.
   1. Expand the Study subfolder in the Projects view and double click the study called StrongFirewallsStud
   2. Select Study in the Component Category in the bottom pane and and select the "Next >" button at the bottom of the dialog.
   3. Compile and close the study by clicking on the File->Save and File->Close menu items.

Running the Model[edit]

1. Expand the Solver subfolder in the Projects view and double click the StrongFirewallsSim simulator.
2. Click on the "Run Simulation" tab and click the "Start Simulation" button.
3. Wait for the simulation to complete, this may take several minutes depending on the speed of your machine. When the simulation finishes you should now be looking at the "Results" tab.
4. Scroll down to see the mean values of the performance variable at the various time points.
5. Values should be similar to

_________________________________Mean Results_________________________________
Name                  Time               Mean                     Confidence Interval
GoalAchieved          0.0               0.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          2.0               5.1500000000E-01  +/-   3.0991872098E-02
GoalAchieved          4.0               9.7000000000E-01  +/-   1.0578396025E-02
GoalAchieved          6.0               1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          8.0               1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          10.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          12.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          14.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          16.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          18.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          20.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          22.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          24.0              1.0000000000E00   +/-   0.0000000000E00

Click the File->Close menu item.

According to this model it is certain that the Engineer Insider will gain unauthorized access to the SCADA network 24 hours after beginning the attack with this system configuration. We stress that quantitative metrics calculated by ADVISE models are not intended to stand alone. Rather, these metrics are supposed to be compared to metrics that come from similar models, such as the system configuration with the weak firewalls created earlier in the tutorial.

Interpreting the Results[edit]

According to this model, the adversary gains access to the SCADA network with a probability of approximately 0.61 after 24 hours, given a system configuration with firewalls with weak administrator authentication. However, the adversary gains access to the SCADA network with a probability of 1 after 24 hours, given a system configuration with firewalls with strong administrator authentication. This suggests that investing in firewalls with strong administrator authentication is not sufficient to improve the security posture the utility when facing an insider engineer adversary, as the adversary may simply attempt a different avenue of attack the bypasses the firewalls entirely.