ADVISE Meta Alpha Tutorial

From Mobius Wiki
Revision as of 20:51, 15 August 2016 by Kjkeefe (talk | contribs) (→‎Creating global variables)
Jump to: navigation, search

Return to Workshop Home

This page is home for the tutorial demonstrating the ADVISE Meta Alpha Tool.

Background

The goal of this tutorial is to demonstrate the ADVISE Meta Tool. We create two models of an electric distribution system, one with firewalls with weak administrator authentication and one with firewalls with strong administrator authentication, and compare their vulnerability in the face of an attack by an insider. The quantitative security evaluation enabled by this tool may help a system architect select the appropriate design for their security posture.

In this tutorial we will build the model from scratch. Here is a completed version of the project built using this tutorial.

Creating a Model for Security Evaluation Using the ADVISE Meta Tool

The first time you run the alpha tool, you may be prompted for your Mobius license. Enter your Mobius user account or select the license file you have downloaded.

Importing an Ontology

File:AMT-ontTree.png
Ontology tree after importing ontology.

Before we can start creating ADVISE Meta models, we need an ontology to work with. Download the ontology used for this tutorial.

The first time you open Mobius, the Ontology view may not be visible. Show it by selecting the menu Window -> Show View -> Ontology Explorer.

To import the ontology

  1. Right click on the white space in the Ontology view.
  2. Select Import...
  3. Navigate to the ontology you downloaded previously, select it and press Open.

Creating an ADVISE Meta Model

Creating a new project

  1. Switch from the Ontology view to the Projects view by click on the Projects tab in the upper left corner of the window.
  2. Right click the Open Projects folder and select New Project.
  3. Name the project AdviseMetaTutorial.

Creating global variables

File:AMT-globVars.png
The global variables defined in the Project properties window.

Global variables are model parameters that we can use to define experiments on our system by varying the initial values of the global variables. We need to create two global variables for this tutorial.

  1. Right click on the AdviseMetaTutorial project folder, and click Properties.
  2. Double click the Global Variables item in the list.
  3. Add a global variable of type character called userAuthType.
    1. Click Add.
    2. Enter userAuthType for the name.
    3. Select Character from the Type drop down menu.
    4. Click Finish.
  4. Add a global variable of type short called userAuthStrength
    1. Click Add.
    2. Enter userAuthStrength for the name.
    3. Leave Short selected in the Type drop down menu.
    4. Click Finish.
  5. Click Apply, then click OK to exit.
This is an example of the defined global variables.

Creating a New ADVISE Meta Atomic Model

  1. Right click on the AdviseMetaTutorial project folder and select New.
  2. Select Atomic in the bottom pane and click Next.
  3. Select the ADVISE Meta Model from the list, enter the name MetaModel, and click Finish.

Creating a System Model

The system model represents the components of the system of interest and the relationships among these components. The components may include (but are not limited to) networks, firewalls, and workstations, for example. The onNetwork relationship is an example of a relationship that may exist between a workstation and a network.

Creating Networks
  1. In the components tree on the left, expand the PhysicalThing component node to make the Device and Network component types visible.
  2. Add a Network component called EngrLAN
    1. Drag and drop a Network component on to the diagram.
    2. Click on the new Network 1 component and select the Edit Details button.
    3. Change the name to EngrLAN, but leave the other attributes set to the defaults.
    4. Click Finish.
  3. In the components tree on the left, expand the Network component node to make the WiredNetwork and WirelessNetwork component types visible.
  4. Add a WiredNetwork component called CorpLAN and leave the attributes set to the defaults.
  5. Add a WiredNetwork component called SCADALAN and leave the attributes set to the defaults.
Creating Firewalls
  1. In the components tree on the left, expand the Device component node under the PhysicalThing node to make the Firewall and Host component types visible, then expand the Firewall component node to make the FirewallAppliance and FirewallHosted component types visible.
  2. Add a FirewallHosted and name it CorpLanScadaLanFW. Enter userAuthStrength in the strengthOfUserAuthentication textbox, and enter userAuthType in the userAuthenticationType textbox.
    This is an example of a completed firewall component.
  3. Add another FirewallHosted and name it CorpLanEngrLanFW. Enter userAuthStrength in the strengthOfUserAuthentication textbox, and enter userAuthType in the userAuthenticationType textbox.
  4. Add another FirewallHosted and name it EngrLanScadaLanFW. Change strengthOfUserAuthentication from 0 to 1, and change userAuthenticationType from N to W.
  5. Add a new onNetwork relationship from the CorpLanScadaLanFW to the corpLAN component.
    1. Select the CorpLanScadaLanFW component, click Add Relationship, and click on the corpLAN component.
    2. Select the onNetwork relationship from the drop down list.
    3. Click Finish.
  6. Add a new onNetwork relationship from the CorpLanScadaLanFW to the SCADALAN component.
  7. Add a new onNetwork relationship from the EngrLanScadaLanFW to the SCADALAN component.
  8. Add a new onNetwork relationship from the EngrLanScadaLanFW to the EngrLAN component.
  9. Add a new onNetwork relationship from the CorpLanEngrLanFW to the EngrLAN component.
  10. Add a new onNetwork relationship from the CorpLanEngrLanFW to the CorpLAN component.
Creating an SSH Server on a Linux Workstation
  1. In the components tree on the left, expand the Host component node to make the FirewallHosted, Server and Workstation component types visible.
  2. Add a new Workstation and name it EngrWorkstation.
  3. Add a new onNetwork relationship from the EngrWorkstation to the EngrLAN component.
  4. In the components tree on the left, expand the System component node to make the Device and Software component types visible, then expand the Software component node to make Application and OperatingSystem visible.
  5. Add a new OperatingSystem and name it LinuxOS.
  6. Add a new hardwarePlatform relationship from the LinuxOS to the EngrWorkstation, in the same way you created the onNetwork relationships.
  7. Add a new Application and name it SSHServer.
  8. Add a new applicationOS relationship from the SSHServer to the LinuxOS, in the same way you created the onNetwork and hardwarePlatform relationships.
Creating an HMI on the SCADALAN
  1. Add a new Host and name it HMI.
  2. Add a new onNetwork relationship from the HMI to the SCADALAN component.
Finish the System Model
  1. Click the File -> Save menu item.
This is an example of the complete ADVISE Meta system diagram.

Defining the Goals

  1. Select the Goals tab at the bottom of the ADVISE Meta Atomic Model Editor window.
  2. Select the New Goal 1 goal from the list in the upper right.
  3. Change the name below to read Goal_GainScadaNetworkAccess
  4. Add the SCADALAN_NetworkAccess state variable to the list of state variables
    1. Expand the SCADALAN component in the Available State Variables tree on the left side of the window.
    2. Click and drag the NetworkAccess state variable, and drop it in the list of Dependent State Variables.
  5. Change the text in the Goal Expression to the following code: 
    return SCADALAN_NetworkAccess->Mark();
  6. Click the File -> Save menu item.
This is an example of the completed goal definition.

Creating an Adversary Model

  1. Select the Adversaries tab at the bottom of the ADVISE Meta Atomic Model Editor window.
  2. Click and drag the IndependentInsider adversary template over to the Adversaries list on the right side.
  3. Change the name below to EngineerInsider.
  4. Change Cost of Detection to 100000.
  5. In the Access section, add the access elements the adversary holds at the beginning of the attack.
    1. Click the Add... button.
    2. Select the CorpLAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    3. Select the CorpLanEngrLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    4. Select the CorpLanScadaLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    5. Select the EngrLAN_NetworkAccess element in the left pane, and click the > button to move it to the right pane.
    6. Select the EngrLAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    7. Select the EngrLanScadaLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    8. Select the EngrWorkstation_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
    9. Select the EngrWorkstation_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
    10. Select the EngrWorkstation_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    11. Select the EngrWorkstation_UIAccess element in the left pane, and click the > button to move it to the right pane.
    12. Select the HMI_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    13. Select the LinuxOS_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
    14. Select the LinuxOS_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
    15. Select the LinuxOS_UIAccess element in the left pane, and click the > button to move it to the right pane.
    16. Select the SCADALAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    17. Select the SSHServer_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
    18. Select the SSHServer_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
    19. Select the SSHServer_UIAccess element in the left pane, and click the > button to move it to the right pane.
    20. Click Finish.
  6. The adversary does not start the attack with any special knowledge of the system.
  7. Customize the adversary's skill level in the Skills section.
    1. Click the Add... button.
    2. Change the initial value of the KineticEquipmentAttack skill to 750.
    3. Click Finish
  8. In the Goals section, add the Goal_GainNetworkAccessOnScadaNetwork goal.
    1. Click the Add.... button.
    2. Select the Goal_GainScadaNetworkAccess element in the left pane, and click the > button to move it to the right pane.
    3. Select the Goal_GainScadaNetworkAccess element, and then press the Payoff cell and enter the value 50000.
    4. Click Finish.
  9. Click the File -> Save menu item.

Creating a Weak Firewalls Configuration

  1. Select the Configurations tab at the bottom of the ADVISE Meta Atomic Model Editor window.
  2. Select the New Configuration 1 configuration from the list on the left.
  3. Change the name on the right side to read WeakFirewalls
  4. Add the goal defined previously to the configuration.
    1. Click the Add button in the Goals section.
    2. Select the Goal_GainScadaNetworkAccess goal from the list.
    3. Click Finish.
  5. Select EngineerInsider as the adversary for this configuration from the drop down list.
  6. Enter 1 as the Value for userAuthStrength in the Global Variables pane.
  7. Enter 'W' (including the apostrophes) as the Value for userAuthType in the Global Variables pane.
  8. Click the File -> Save menu item.
This is an example of the completed configuration.

Creating a Strong Firewalls Configuration

  1. Click on the Add button in the upper left of the screen to add another configuration.
  2. Change the name on the right side to read StrongFirewalls
  3. Add the goal defined previously to the configuration.
    1. Click the Add button in the Goals section.
    2. Select the Goal_GainScadaNetworkAccess goal from the list.
    3. Click Finish.
  4. Select EngineerInsider as the adversary for this configuration from the drop down list.
  5. Enter 8 as the Value for userAuthStrength in the Global Variables pane.
  6. Enter 'T' (including the apostrophes) as the Value for userAuthType in the Global Variables pane.
  7. Click the File -> Save menu item.

Generating an ADVISE Model

To generate the ADVISE Model:

  1. Select the Generator tab at the bottom of the ADVISE Meta Atomic Model Editor window.
  2. Select the WeakFirewalls and StrongFirewalls configurations by shift-clicking from the list on the left.
  3. Click the Generate button. Be patient, this process may take a couple of minutes.
  4. After getting confirmation of the creation of the simulator, Click on the File->Close menu item.

Defining the Reward Model

We will define a performance variable that will track whether the adversary achieved the goal over a 24-hour period.

  1. Expand the Reward subfolder in the projects view to reveal newADVISEModelPVs model and double click it to open it.
  2. Add a new variable called GoalAchieved by entering "GoalAchieved" into the textbox in the top left of the dialog and then pressing "Add Variable:".
  3. In the "Rate Rewards" tab, define the reward function to be:
    return newADVISEModel->Goal_GainScadaNetworkAccess->Mark();;
  4. Click on the "Apply Changes" button.
  5. In the "Time" tab, define the timing type to be an "Instant of Time" with an incremental range. The range should be 0-60 with a step size of 10.
    1. Select the Time Tab
    2. From the Type drop down list select Instant of Time.
    3. From the Time Point definition method select Incremental Range.
    4. In the First time point in series textbox enter 0.0
    5. In the Upper Bound of series textbox enter 24.0
    6. In the Step size in series textbox enter 2.0
    7. Click on the "Apply Changes" button.
  6. Save and close the reward model by clicking File->Save and File->Close.

This defines a reward model for your ADVISE atomic model that measures the state of the goal over a 24-hour period.

Running the Model and Viewing the Results

  1. Expand the Solver subfolder in the Projects view and double click the newADVISEModelSim simulator.
  2. Click on the "Run Simulation" tab and click the "Start Simulation" button.
  3. Wait for the simulation to complete, this may take several minutes depending on the speed of your machine. When the simulation finishes you should now be looking at the "Results" tab.
  4. Scroll down to see the mean values of the performance variable at the various time points.
  5. Values should be similar to 
_________________________________Experiment 1_________________________________
Start Time:                  Sun Aug 14 22:21:37 CDT 2016
Finish Time:                 Sun Aug 14 22:22:26 CDT 2016
Elapsed Running Time:        48.419
Total CPU Time:              31.167
Batches Completed:           1000
Experiment Name:             StrongFirewalls
Global Variable Settings:    
        Short                userAuthStrength         8                      
        Character            userAuthType             'T'                    


_________________________________Mean Results_________________________________
Name                  Time               Mean                     Confidence Interval
GoalAchieved          0.0               0.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          2.0               5.1500000000E-01  +/-   3.0991872098E-02
GoalAchieved          4.0               9.7000000000E-01  +/-   1.0578396025E-02
GoalAchieved          6.0               1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          8.0               1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          10.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          12.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          14.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          16.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          18.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          20.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          22.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          24.0              1.0000000000E00   +/-   0.0000000000E00 


_______________________________Variance Results_______________________________


_______________________________Interval Results_______________________________


_____________________________Distribution Results_____________________________

_________________________________Experiment 2_________________________________
Start Time:                  Sun Aug 14 22:22:26 CDT 2016
Finish Time:                 Sun Aug 14 22:23:25 CDT 2016
Elapsed Running Time:        59.484
Total CPU Time:              40.158
Batches Completed:           10000
Experiment Name:             WeakFirewalls
Global Variable Settings:    
        Short                userAuthStrength         1                      
        Character            userAuthType             'W'                    


_________________________________Mean Results_________________________________
Name                  Time               Mean                     Confidence Interval
GoalAchieved          0.0               0.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          2.0               0.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          4.0               3.0000000000E-03  +/-   1.0719782991E-03  (*)
GoalAchieved          6.0               3.6300000000E-02  +/-   3.6660807717E-03  (*)
GoalAchieved          8.0               1.6290000000E-01  +/-   7.2381403552E-03
GoalAchieved          10.0              3.4830000000E-01  +/-   9.3385271315E-03
GoalAchieved          12.0              4.9200000000E-01  +/-   9.7992354937E-03
GoalAchieved          14.0              5.8290000000E-01  +/-   9.6648453634E-03
GoalAchieved          16.0              6.6230000000E-01  +/-   9.2698070620E-03
GoalAchieved          18.0              7.3310000000E-01  +/-   8.6702911980E-03
GoalAchieved          20.0              7.8630000000E-01  +/-   8.0347836302E-03
GoalAchieved          22.0              8.2660000000E-01  +/-   7.4207881630E-03
GoalAchieved          24.0              8.6220000000E-01  +/-   6.7562607951E-03


_______________________________Variance Results_______________________________


_______________________________Interval Results_______________________________


_____________________________Distribution Results_____________________________


(*) This result did not converge to within the specified confidence interval


Click the File->Close menu item.

To see the adversary paths.

  1. Open the newADVISEModel ADVISE model under the Atomic folder.
  2. Press View
  3. Press Clear
  4. Press Show Results

Scroll around the Attack Execution Graph to see the adversary paths.

This is an example of the path chosen by the adversary through the strong firewall configuration.
This is an example of the path chosen by the adversary through the weak firewall configuration.

From these images we can derive the attack paths. When faced with the Strong Firewalls configuration the adversary follows this attack path:

  1. HMI_GainLocalUIAccessDevice
  2. SCADALAN_GainNetworkAccessViaNetworkNode

When faced with the Weak Firewalls configuration the adversary follows this attack path:

  1. EngrLanScadaLanFW_GainRemoteUIAccessDevice
  2. EngrLanScadaLanFW_GainUserCredentials
  3. EngrLanScadaLanFW_GainLogicalAccess
  4. SCADALAN_GainNetworkAccessViaNetworkNode

Interpreting the Results

According to this model, the adversary gains access to the SCADA network with a probability of approximately 0.86 after 24 hours, given a system configuration with firewalls with weak administrator authentication. However, the adversary gains access to the SCADA network with a probability of 1 after 24 hours, given a system configuration with firewalls with strong administrator authentication.

This result may not agree with our intuition. We expect that strengthening the defenses will slow the adversary. However, the model says that the adversary achieves his or her goal faster when the defenses are strengthened. However, this surprising result can be explained. The adversary does not factor the speed of completing a set of attacks in this model, but only its net expected profitability (the expected payoff minus the expected cost). Though the adversary can achieve the goal faster with strengthened defenses, it costs more. Our results, presented below, show that with weak defenses the adversary has an net expected profit of 43090.2, but with the strengthened defenses the adversary has an expected net profit of only 35821.6. Strengthening the defenses reduces the adversary's expected profit by (43090.2 - 35821.6) = 7268.6.

This result suggests that investing in firewalls with strong administrator authentication is not sufficient to improve the security posture of the utility when facing an insider engineer adversary, as the adversary may simply attempt a different avenue of attack that bypasses the firewalls entirely.

The net profitability of the most attractive paths from an arbitrary starting point in the Strong Firewalls Configuration 

HMI_GainLocalUIAccessDevice <- Attack Chosen by the Adversary
     Total Cost for                  HMI_GainLocalUIAccessDevice is 2
     Total Detection Probability for HMI_GainLocalUIAccessDevice is 0.141764
     Total Payoff for                HMI_GainLocalUIAccessDevice is 50000
  Attractiveness for HMI_GainLocalUIAccessDevice is 35821.6



EngrLanScadaLanFW_GainRemoteUIAccessDevice
     Total Cost for                  EngrLanScadaLanFW_GainRemoteUIAccessDevice is 3
     Total Detection Probability for EngrLanScadaLanFW_GainRemoteUIAccessDevice is 0.141764
     Total Payoff for                EngrLanScadaLanFW_GainRemoteUIAccessDevice is 50000
  Attractiveness for EngrLanScadaLanFW_GainRemoteUIAccessDevice is 35820.6

The net profitability of the most attractive paths starting with a particular individual attack in the Weak Firewalls Configuration 

HMI_GainLocalUIAccessDevice
     Total Cost for                  HMI_GainLocalUIAccessDevice is 2
     Total Detection Probability for HMI_GainLocalUIAccessDevice is 0.141764
     Total Payoff for                HMI_GainLocalUIAccessDevice is 50000
  Attractiveness for HMI_GainLocalUIAccessDevice is 35821.6



EngrLanScadaLanFW_GainRemoteUIAccessDevice <- Attack Chosen by the Adversary
     Total Cost for                  EngrLanScadaLanFW_GainRemoteUIAccessDevice is 803.4
     Total Detection Probability for EngrLanScadaLanFW_GainRemoteUIAccessDevice is 0.0610645
     Total Payoff for                EngrLanScadaLanFW_GainRemoteUIAccessDevice is 50000
  Attractiveness for EngrLanScadaLanFW_GainRemoteUIAccessDevice is 43090.2

Return to Workshop Home

This page is home for the tutorial demonstrating the ADVISE Meta Alpha Tool.

Background[edit]

The goal of this tutorial is to demonstrate the ADVISE Meta Tool. We create two models of an electric distribution system, one with firewalls with weak administrator authentication and one with firewalls with strong administrator authentication, and compare their vulnerability in the face of an attack by an insider. The quantitative security evaluation enabled by this tool may help a system architect select the appropriate design for their security posture.

In this tutorial we will build the model from scratch. Here is a completed version of the project built using this tutorial.

Creating a Model for Security Evaluation Using the ADVISE Meta Tool[edit]

The first time you run the alpha tool, you may be prompted for your Mobius license. Enter your Mobius user account or select the license file you have downloaded.

Importing an Ontology[edit]

File:AMT-ontTree.png
Ontology tree after importing ontology.

Before we can start creating ADVISE Meta models, we need an ontology to work with. Download the ontology used for this tutorial.

The first time you open Mobius, the Ontology view may not be visible. Show it by selecting the menu Window -> Show View -> Ontology Explorer.

To import the ontology

  1. Right click on the white space in the Ontology view.
  2. Select Import...
  3. Navigate to the ontology you downloaded previously, select it and press Open.

Creating an ADVISE Meta Model[edit]

Creating a new project[edit]

  1. Switch from the Ontology view to the Projects view by click on the Projects tab in the upper left corner of the window.
  2. Right click the Open Projects folder and select New Project.
  3. Name the project AdviseMetaTutorial.

Creating global variables[edit]

File:AMT-globVars.png
The global variables defined in the Project properties window.

Global variables are model parameters that we can use to define experiments on our system by varying the initial values of the global variables. We need to create two global variables for this tutorial.

  1. Right click on the AdviseMetaTutorial project folder, and click Properties.
  2. Double click the Global Variables item in the list.
  3. Add a global variable of type character called userAuthType.
    1. Click Add.
    2. Enter userAuthType for the name.
    3. Select Character from the Type drop down menu.
    4. Click Finish.
  4. Add a global variable of type short called userAuthStrength
    1. Click Add.
    2. Enter userAuthStrength for the name.
    3. Leave Short selected in the Type drop down menu.
    4. Click Finish.
  5. Click Apply, then click OK to exit.
This is an example of the defined global variables.

Creating a New ADVISE Meta Atomic Model[edit]

  1. Right click on the AdviseMetaTutorial project folder and select New.
  2. Select Atomic in the bottom pane and click Next.
  3. Select the ADVISE Meta Model from the list, enter the name MetaModel, and click Finish.

Creating a System Model[edit]

The system model represents the components of the system of interest and the relationships among these components. The components may include (but are not limited to) networks, firewalls, and workstations, for example. The onNetwork relationship is an example of a relationship that may exist between a workstation and a network.

Creating Networks[edit]
  1. In the components tree on the left, expand the PhysicalThing component node to make the Device and Network component types visible.
  2. Add a Network component called EngrLAN
    1. Drag and drop a Network component on to the diagram.
    2. Click on the new Network 1 component and select the Edit Details button.
    3. Change the name to EngrLAN, but leave the other attributes set to the defaults.
    4. Click Finish.
  3. In the components tree on the left, expand the Network component node to make the WiredNetwork and WirelessNetwork component types visible.
  4. Add a WiredNetwork component called CorpLAN and leave the attributes set to the defaults.
  5. Add a WiredNetwork component called SCADALAN and leave the attributes set to the defaults.
Creating Firewalls[edit]
  1. In the components tree on the left, expand the Device component node under the PhysicalThing node to make the Firewall and Host component types visible, then expand the Firewall component node to make the FirewallAppliance and FirewallHosted component types visible.
  2. Add a FirewallHosted and name it CorpLanScadaLanFW. Enter userAuthStrength in the strengthOfUserAuthentication textbox, and enter userAuthType in the userAuthenticationType textbox.
    This is an example of a completed firewall component.
  3. Add another FirewallHosted and name it CorpLanEngrLanFW. Enter userAuthStrength in the strengthOfUserAuthentication textbox, and enter userAuthType in the userAuthenticationType textbox.
  4. Add another FirewallHosted and name it EngrLanScadaLanFW. Change strengthOfUserAuthentication from 0 to 1, and change userAuthenticationType from N to W.
  5. Add a new onNetwork relationship from the CorpLanScadaLanFW to the corpLAN component.
    1. Select the CorpLanScadaLanFW component, click Add Relationship, and click on the corpLAN component.
    2. Select the onNetwork relationship from the drop down list.
    3. Click Finish.
  6. Add a new onNetwork relationship from the CorpLanScadaLanFW to the SCADALAN component.
  7. Add a new onNetwork relationship from the EngrLanScadaLanFW to the SCADALAN component.
  8. Add a new onNetwork relationship from the EngrLanScadaLanFW to the EngrLAN component.
  9. Add a new onNetwork relationship from the CorpLanEngrLanFW to the EngrLAN component.
  10. Add a new onNetwork relationship from the CorpLanEngrLanFW to the CorpLAN component.
Creating an SSH Server on a Linux Workstation[edit]
  1. In the components tree on the left, expand the Host component node to make the FirewallHosted, Server and Workstation component types visible.
  2. Add a new Workstation and name it EngrWorkstation.
  3. Add a new onNetwork relationship from the EngrWorkstation to the EngrLAN component.
  4. In the components tree on the left, expand the System component node to make the Device and Software component types visible, then expand the Software component node to make Application and OperatingSystem visible.
  5. Add a new OperatingSystem and name it LinuxOS.
  6. Add a new hardwarePlatform relationship from the LinuxOS to the EngrWorkstation, in the same way you created the onNetwork relationships.
  7. Add a new Application and name it SSHServer.
  8. Add a new applicationOS relationship from the SSHServer to the LinuxOS, in the same way you created the onNetwork and hardwarePlatform relationships.
Creating an HMI on the SCADALAN[edit]
  1. Add a new Host and name it HMI.
  2. Add a new onNetwork relationship from the HMI to the SCADALAN component.
Finish the System Model[edit]
  1. Click the File -> Save menu item.
This is an example of the complete ADVISE Meta system diagram.

Defining the Goals[edit]

  1. Select the Goals tab at the bottom of the ADVISE Meta Atomic Model Editor window.
  2. Select the New Goal 1 goal from the list in the upper right.
  3. Change the name below to read Goal_GainScadaNetworkAccess
  4. Add the SCADALAN_NetworkAccess state variable to the list of state variables
    1. Expand the SCADALAN component in the Available State Variables tree on the left side of the window.
    2. Click and drag the NetworkAccess state variable, and drop it in the list of Dependent State Variables.
  5. Change the text in the Goal Expression to the following code: 
    return SCADALAN_NetworkAccess->Mark();
  6. Click the File -> Save menu item.
This is an example of the completed goal definition.

Creating an Adversary Model[edit]

  1. Select the Adversaries tab at the bottom of the ADVISE Meta Atomic Model Editor window.
  2. Click and drag the IndependentInsider adversary template over to the Adversaries list on the right side.
  3. Change the name below to EngineerInsider.
  4. Change Cost of Detection to 100000.
  5. In the Access section, add the access elements the adversary holds at the beginning of the attack.
    1. Click the Add... button.
    2. Select the CorpLAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    3. Select the CorpLanEngrLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    4. Select the CorpLanScadaLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    5. Select the EngrLAN_NetworkAccess element in the left pane, and click the > button to move it to the right pane.
    6. Select the EngrLAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    7. Select the EngrLanScadaLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    8. Select the EngrWorkstation_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
    9. Select the EngrWorkstation_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
    10. Select the EngrWorkstation_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    11. Select the EngrWorkstation_UIAccess element in the left pane, and click the > button to move it to the right pane.
    12. Select the HMI_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    13. Select the LinuxOS_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
    14. Select the LinuxOS_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
    15. Select the LinuxOS_UIAccess element in the left pane, and click the > button to move it to the right pane.
    16. Select the SCADALAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
    17. Select the SSHServer_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
    18. Select the SSHServer_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
    19. Select the SSHServer_UIAccess element in the left pane, and click the > button to move it to the right pane.
    20. Click Finish.
  6. The adversary does not start the attack with any special knowledge of the system.
  7. Customize the adversary's skill level in the Skills section.
    1. Click the Add... button.
    2. Change the initial value of the KineticEquipmentAttack skill to 750.
    3. Click Finish
  8. In the Goals section, add the Goal_GainNetworkAccessOnScadaNetwork goal.
    1. Click the Add.... button.
    2. Select the Goal_GainScadaNetworkAccess element in the left pane, and click the > button to move it to the right pane.
    3. Select the Goal_GainScadaNetworkAccess element, and then press the Payoff cell and enter the value 50000.
    4. Click Finish.
  9. Click the File -> Save menu item.

Creating a Weak Firewalls Configuration[edit]

  1. Select the Configurations tab at the bottom of the ADVISE Meta Atomic Model Editor window.
  2. Select the New Configuration 1 configuration from the list on the left.
  3. Change the name on the right side to read WeakFirewalls
  4. Add the goal defined previously to the configuration.
    1. Click the Add button in the Goals section.
    2. Select the Goal_GainScadaNetworkAccess goal from the list.
    3. Click Finish.
  5. Select EngineerInsider as the adversary for this configuration from the drop down list.
  6. Enter 1 as the Value for userAuthStrength in the Global Variables pane.
  7. Enter 'W' (including the apostrophes) as the Value for userAuthType in the Global Variables pane.
  8. Click the File -> Save menu item.
This is an example of the completed configuration.

Creating a Strong Firewalls Configuration[edit]

  1. Click on the Add button in the upper left of the screen to add another configuration.
  2. Change the name on the right side to read StrongFirewalls
  3. Add the goal defined previously to the configuration.
    1. Click the Add button in the Goals section.
    2. Select the Goal_GainScadaNetworkAccess goal from the list.
    3. Click Finish.
  4. Select EngineerInsider as the adversary for this configuration from the drop down list.
  5. Enter 8 as the Value for userAuthStrength in the Global Variables pane.
  6. Enter 'T' (including the apostrophes) as the Value for userAuthType in the Global Variables pane.
  7. Click the File -> Save menu item.

Generating an ADVISE Model[edit]

To generate the ADVISE Model:

  1. Select the Generator tab at the bottom of the ADVISE Meta Atomic Model Editor window.
  2. Select the WeakFirewalls and StrongFirewalls configurations by shift-clicking from the list on the left.
  3. Click the Generate button. Be patient, this process may take a couple of minutes.
  4. After getting confirmation of the creation of the simulator, Click on the File->Close menu item.

Defining the Reward Model[edit]

We will define a performance variable that will track whether the adversary achieved the goal over a 24-hour period.

  1. Expand the Reward subfolder in the projects view to reveal newADVISEModelPVs model and double click it to open it.
  2. Add a new variable called GoalAchieved by entering "GoalAchieved" into the textbox in the top left of the dialog and then pressing "Add Variable:".
  3. In the "Rate Rewards" tab, define the reward function to be:
    return newADVISEModel->Goal_GainScadaNetworkAccess->Mark();;
  4. Click on the "Apply Changes" button.
  5. In the "Time" tab, define the timing type to be an "Instant of Time" with an incremental range. The range should be 0-60 with a step size of 10.
    1. Select the Time Tab
    2. From the Type drop down list select Instant of Time.
    3. From the Time Point definition method select Incremental Range.
    4. In the First time point in series textbox enter 0.0
    5. In the Upper Bound of series textbox enter 24.0
    6. In the Step size in series textbox enter 2.0
    7. Click on the "Apply Changes" button.
  6. Save and close the reward model by clicking File->Save and File->Close.

This defines a reward model for your ADVISE atomic model that measures the state of the goal over a 24-hour period.

Running the Model and Viewing the Results[edit]

  1. Expand the Solver subfolder in the Projects view and double click the newADVISEModelSim simulator.
  2. Click on the "Run Simulation" tab and click the "Start Simulation" button.
  3. Wait for the simulation to complete, this may take several minutes depending on the speed of your machine. When the simulation finishes you should now be looking at the "Results" tab.
  4. Scroll down to see the mean values of the performance variable at the various time points.
  5. Values should be similar to 
_________________________________Experiment 1_________________________________
Start Time:                  Sun Aug 14 22:21:37 CDT 2016
Finish Time:                 Sun Aug 14 22:22:26 CDT 2016
Elapsed Running Time:        48.419
Total CPU Time:              31.167
Batches Completed:           1000
Experiment Name:             StrongFirewalls
Global Variable Settings:    
        Short                userAuthStrength         8                      
        Character            userAuthType             'T'                    


_________________________________Mean Results_________________________________
Name                  Time               Mean                     Confidence Interval
GoalAchieved          0.0               0.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          2.0               5.1500000000E-01  +/-   3.0991872098E-02
GoalAchieved          4.0               9.7000000000E-01  +/-   1.0578396025E-02
GoalAchieved          6.0               1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          8.0               1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          10.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          12.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          14.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          16.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          18.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          20.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          22.0              1.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          24.0              1.0000000000E00   +/-   0.0000000000E00 


_______________________________Variance Results_______________________________


_______________________________Interval Results_______________________________


_____________________________Distribution Results_____________________________

_________________________________Experiment 2_________________________________
Start Time:                  Sun Aug 14 22:22:26 CDT 2016
Finish Time:                 Sun Aug 14 22:23:25 CDT 2016
Elapsed Running Time:        59.484
Total CPU Time:              40.158
Batches Completed:           10000
Experiment Name:             WeakFirewalls
Global Variable Settings:    
        Short                userAuthStrength         1                      
        Character            userAuthType             'W'                    


_________________________________Mean Results_________________________________
Name                  Time               Mean                     Confidence Interval
GoalAchieved          0.0               0.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          2.0               0.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          4.0               3.0000000000E-03  +/-   1.0719782991E-03  (*)
GoalAchieved          6.0               3.6300000000E-02  +/-   3.6660807717E-03  (*)
GoalAchieved          8.0               1.6290000000E-01  +/-   7.2381403552E-03
GoalAchieved          10.0              3.4830000000E-01  +/-   9.3385271315E-03
GoalAchieved          12.0              4.9200000000E-01  +/-   9.7992354937E-03
GoalAchieved          14.0              5.8290000000E-01  +/-   9.6648453634E-03
GoalAchieved          16.0              6.6230000000E-01  +/-   9.2698070620E-03
GoalAchieved          18.0              7.3310000000E-01  +/-   8.6702911980E-03
GoalAchieved          20.0              7.8630000000E-01  +/-   8.0347836302E-03
GoalAchieved          22.0              8.2660000000E-01  +/-   7.4207881630E-03
GoalAchieved          24.0              8.6220000000E-01  +/-   6.7562607951E-03


_______________________________Variance Results_______________________________


_______________________________Interval Results_______________________________


_____________________________Distribution Results_____________________________


(*) This result did not converge to within the specified confidence interval


Click the File->Close menu item.

To see the adversary paths.

  1. Open the newADVISEModel ADVISE model under the Atomic folder.
  2. Press View
  3. Press Clear
  4. Press Show Results

Scroll around the Attack Execution Graph to see the adversary paths.

This is an example of the path chosen by the adversary through the strong firewall configuration.
This is an example of the path chosen by the adversary through the weak firewall configuration.

From these images we can derive the attack paths. When faced with the Strong Firewalls configuration the adversary follows this attack path:

  1. HMI_GainLocalUIAccessDevice
  2. SCADALAN_GainNetworkAccessViaNetworkNode

When faced with the Weak Firewalls configuration the adversary follows this attack path:

  1. EngrLanScadaLanFW_GainRemoteUIAccessDevice
  2. EngrLanScadaLanFW_GainUserCredentials
  3. EngrLanScadaLanFW_GainLogicalAccess
  4. SCADALAN_GainNetworkAccessViaNetworkNode

Interpreting the Results[edit]

According to this model, the adversary gains access to the SCADA network with a probability of approximately 0.86 after 24 hours, given a system configuration with firewalls with weak administrator authentication. However, the adversary gains access to the SCADA network with a probability of 1 after 24 hours, given a system configuration with firewalls with strong administrator authentication.

This result may not agree with our intuition. We expect that strengthening the defenses will slow the adversary. However, the model says that the adversary achieves his or her goal faster when the defenses are strengthened. However, this surprising result can be explained. The adversary does not factor the speed of completing a set of attacks in this model, but only its net expected profitability (the expected payoff minus the expected cost). Though the adversary can achieve the goal faster with strengthened defenses, it costs more. Our results, presented below, show that with weak defenses the adversary has an net expected profit of 43090.2, but with the strengthened defenses the adversary has an expected net profit of only 35821.6. Strengthening the defenses reduces the adversary's expected profit by (43090.2 - 35821.6) = 7268.6.

This result suggests that investing in firewalls with strong administrator authentication is not sufficient to improve the security posture of the utility when facing an insider engineer adversary, as the adversary may simply attempt a different avenue of attack that bypasses the firewalls entirely.

The net profitability of the most attractive paths from an arbitrary starting point in the Strong Firewalls Configuration 

HMI_GainLocalUIAccessDevice <- Attack Chosen by the Adversary
     Total Cost for                  HMI_GainLocalUIAccessDevice is 2
     Total Detection Probability for HMI_GainLocalUIAccessDevice is 0.141764
     Total Payoff for                HMI_GainLocalUIAccessDevice is 50000
  Attractiveness for HMI_GainLocalUIAccessDevice is 35821.6



EngrLanScadaLanFW_GainRemoteUIAccessDevice
     Total Cost for                  EngrLanScadaLanFW_GainRemoteUIAccessDevice is 3
     Total Detection Probability for EngrLanScadaLanFW_GainRemoteUIAccessDevice is 0.141764
     Total Payoff for                EngrLanScadaLanFW_GainRemoteUIAccessDevice is 50000
  Attractiveness for EngrLanScadaLanFW_GainRemoteUIAccessDevice is 35820.6

The net profitability of the most attractive paths starting with a particular individual attack in the Weak Firewalls Configuration 

HMI_GainLocalUIAccessDevice
     Total Cost for                  HMI_GainLocalUIAccessDevice is 2
     Total Detection Probability for HMI_GainLocalUIAccessDevice is 0.141764
     Total Payoff for                HMI_GainLocalUIAccessDevice is 50000
  Attractiveness for HMI_GainLocalUIAccessDevice is 35821.6



EngrLanScadaLanFW_GainRemoteUIAccessDevice <- Attack Chosen by the Adversary
     Total Cost for                  EngrLanScadaLanFW_GainRemoteUIAccessDevice is 803.4
     Total Detection Probability for EngrLanScadaLanFW_GainRemoteUIAccessDevice is 0.0610645
     Total Payoff for                EngrLanScadaLanFW_GainRemoteUIAccessDevice is 50000
  Attractiveness for EngrLanScadaLanFW_GainRemoteUIAccessDevice is 43090.2
Retrieved from "https://www.mobius.illinois.edu/wiki/index.php?title=ADVISE_Meta_Alpha_Tutorial&oldid=1069"