Difference between revisions of "ADVISE Meta Alpha Tutorial"

Return to Workshop Page

Background[edit]

The goal of this tutorial is to demonstrate the ADVISE Meta Tool. We create a model of an electric distribution system, with and without ______________ (attributed difference here), and compare the designs with several different security metrics. The quantitative security evaluation enabled by this tool may help a system architect select the appropriate design for their security posture.

For information on ADVISE, please see Elizabeth LeMay's doctoral thesis, which may be found here,

Addition published papers on ADVISE include:

• State-Based Analysis in ADVISE, published in QEST 2012,
• Implementing the ADVISE Security Modeling Formalism in Mobius, published in DSN 2013,
• Model-based Security Metrics using ADversary VIew Security Evaluation (ADVISE), published in QEST 2011,
• Adversary-Driven State-Based System Security Evaluation, published in MetriSec 2010,
• A Generalized Adversary Decision Algorithm and Analytic Solution Methods for ADVISE Models, Master's Thesis 2012.

Creating a Model for Security Evaluation Using the ADVISE Meta Tool[edit]

The first time you run the alpha tool, you may be prompted for your Mobius license. Enter your Mobius user account.

Importing an Ontology[edit]

Before we can start creating ADVISE Meta models, we need an ontology to work with. You can download the ontology used for this tutorial here: here.

The first time you open Mobius, the Ontology view may not be visible. Show it by selecting the menu Window -> Show View -> Ontology Explorer.

To import the ontology

1. Right click on the white space in the Ontology view.
2. Select Import...
3. Navigate to the ontology you downloaded previously, select it and press Open.

Creating an ADVISE Meta Model[edit]

Creating a New ADVISE Meta Atomic Model[edit]

1. Switch from the Ontology view to the Projects view by click on the Projects tab in the upper left corner of the window.
2. Right click the Open Projects folder and select New Project.
3. Name the project ADVISEMetaTutorial.
4. Right click on the ADVISEMetaTutorial project folder and select New.
5. Select Atomic in the bottom pane and click Next.
6. Select the ADVISE Meta Model from the list, enter the name MetaModel, and click Finish.

Creating a System Model[edit]

The system model represents the components of the system of interest and the relationships among these components. The components may include (but is not limited to) networks, firewalls, and workstations, for example. The onNetwork relationship is an example of a relationship that may exist between a workstation and a network.

Creating Networks[edit]

1. In the components tree on the left, expand the PhysicalThing component node to make the Device and Network component types visible.
2. Add a Network component called EngrLAN
   1. Drag and drop a Network component on to the diagram.
   2. Click on the new Network 1 component and select the Edit Details button.
   3. Change the name to EngrLAN, but leave the other attributes set to the defaults.
   4. Click Finish.
3. In the components tree on the left, expand the Network component node to make the WiredNetwork and WirelessNetwork component types visible.
4. Add a WiredNetwork component called CorpLAN and leave the attributes set to the defaults.
5. Add a WiredNetwork component called SCADALAN and leave the attributes set to the defaults.

Creating Firewalls[edit]

1. In the components tree on the left, expand the Device component node under the PhysicalThing node to make the Firewall and Host component types visible, then expand the Firewall component node to make the FirewallAppliance and FirewallHosted component types visible.
2. Add a FirewallHosted and name it CorpLanScadaLanFW.
3. Add another FirewallHosted and name it CorpLanEngrLanFW.
4. Add another FirewallHosted and name it EngrLanScadaLanFW.
5. Add a new 'onNetwork relationship from the CorpLanScadaLanFW to the corpLAN component.
   1. Select the CorpLanScadaLanFW component, click Add Relationship, and click on the corpLAN component.
   2. Select the onNetwork relationship from the drop down list.
   3. Click Finish.
6. Add a new onNetwork relationship from the CorpLanScadaLanFW to the SCADALAN component.
7. Add a new onNetwork relationship from the EngrLanScadaLanFW to the SCADALAN component.
8. Add a new onNetwork relationship from the EngrLanScadaLanFW to the EngrLAN component.
9. Add a new onNetwork relationship from the CorpLanEngrLanFW to the EngrLAN component.
10. Add a new onNetwork relationship from the CorpLanEngrLanFW to the CorpLAN component.

Creating an SSH Server on a Linux Workstation[edit]

1. In the components tree on the left, expand the Host component node to make the FirewallHosted, Server and Workstation component types visible.
2. Add a new Workstation and name it EngrWorkstation.
3. Add a new onNetwork relationship from the EngrWorkstation to the EngrLAN component.
4. In the components tree on the left, expand the System component node to make the Device and Software component types visible, then expand the Software component node to make Application and OperatingSystem visible.
5. Add a new OperatingSystem and name it LinuxOS.
6. Add a new hardwarePlatform relationship from the LinuxOS to the EngrWorkstation, in the same way you created the onNetwork relationships.
7. Add a new Application and name it SSHServer.
8. Add a new applicationOS relationship from the SSHServer to the LinuxOS, in the same way you created the onNetwork and hardwarePlatform relationships.

Creating an HMI on the SCADALAN[edit]

1. Add a new Host and name it HMI.
2. Add a new onNetwork relationship from the HMI to the SCADALAN component.

Finish the System Model[edit]

1. Click the File -> Save menu item.

This is an example of the complete ADVISE Meta system diagram.

Defining the Goals[edit]

1. Select the Goals tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Select the New Goal 1 goal from the list in the upper right.
3. Change the name below to read Goal_GainNetworkAccessOnScadaNetwork
4. Add the SCADALAN_NetworkAccess state variable to the list of state variables
   1. Expand the SCADALAN component in the Available State Variables tree on the left side of the window.
   2. Click and drag the NetworkAccess state variable, and drop it in the list of Dependent State Variables.
5. Change the text in the Goal Expression to the following code:

   return SCADALAN_NetworkAccess->Mark();
   

6. Click the File -> Save menu item.

Creating an Adversary Model[edit]

1. Select the Adversaries tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Click and drag the IndependentInsider adversary template over to the Adversaries list on the right side.
3. Change the name below to EngineerInsider.
4. Change the planning horizon to 5 and leave the attack preference weights and future discount factors the same as what is defined in the template.
5. In the Access section, add the access elements the adversary holds at the beginning of the attack.
   1. Click the Add button.
   2. Select the CorpLAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   3. Select the EngrLAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   4. Select the SCADALAN_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   5. Select the CorpLanEngrLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   6. Select the EngrLanScadaLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   7. Select the CorpLanScadaLanFW_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   8. Select the EngrWorkstation_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
   9. Select the EngrWorkstation_PhysicalAccess element in the left pane, and click the > button to move it to the right pane.
   10. Select the EngrWorkstation_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
   11. Select the EngrWorkstation_UIAccess element in the left pane, and click the > button to move it to the right pane.
   12. Select the LinuxOS_UIAccess element in the left pane, and click the > button to move it to the right pane.
   13. Select the LinuxOS_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
   14. Select the LinuxOS_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
   15. Select the SSHServer_HasUserCredentials element in the left pane, and click the > button to move it to the right pane.
   16. Select the SSHServer_UIAccess element in the left pane, and click the > button to move it to the right pane.
   17. Select the SSHServer_LogicalAccess element in the left pane, and click the > button to move it to the right pane.
   18. Click Finish.
6. The adversary does not start the attack with any special knowledge of the system, so do not add any knowledge elements.
7. In the Skills section, add additional skills.
   1. Select the EngrWorkstation_SpecializedTechnicalExpertise in the left pane, and click the > button to move it to the right pane
8. Change the initial values of the skills to their proper values.
   1. Select BasicCyberOffense, and then click on the cell with the number .5, and enter 1.
   2. Select NetworkPenetration, and then click on the cell with the number .2, and enter 1.
   3. Click Finish.
9. In the Goals section, add the Goal_GainNetworkAccessOnScadaNetwork goal.
   1. Click the Add button.
   2. Select the Goal_GainNetworkAccessOnScadaNetwork element in the left pane, and click the > button to move it to the right pane.
   3. Select the Goal_GainNetworkAccessOnScadaNetwork element, and then press the Payoff cell and enter the value 1000.
   4. Click Finish.
10. Click the File -> Save menu item.

Defining the Metrics[edit]

TODO Once the metrics work is finished this section should be updated, and the section on creating a reward model should be modified or deleted.

For now, simply progress to the next section.

Creating a Configuration[edit]

1. Select the Configurations tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Select the New Configuration 1 configuration from the list on the left.
3. Change the name on the right side to read SystemConfiguration1
4. Add the goal defined previously to the configuration.
   1. Click the Add button in the Goals section.
   2. Select the Goal_GainNetworkAccessOnScadaNetwork goal from the list.
   3. Click Finish.
5. Select EngineerInsider as the adversary for this configuration from the drop down list.
6. TODO For now leave the metrics blank.
7. Click the File -> Save menu item.

Generating an ADVISE Model[edit]

To generate the ADVISE Model:

1. Select the Generator tab at the bottom of the ADVISE Meta Atomic Model Editor window.
2. Select the SystemConfiguration1 configuration from the list on the left.
3. Click the Generate button.
4. Click on the File->Close menu item.

To compile the ADVISE Mode:

1. Open the newly created ADVISE model called SystemConfiguration1.
2. Click on the File->Save menu button. This will automatically compile the model.
3. After compilation press the File->Close menu item.

Defining the Reward Model[edit]

TODO This section will likely be supplanted by the new ontology-driven metrics approach.

We will add a performance variable that will track whether the adversary achieved the goal.

1. Right click on the ADVISEMetaTutorial project and select "New" from the popup menu.
2. Select "Reward" from the Component Category in the bottom pane and select the "Next >" button at the bottom of the dialog.
3. The component type will be "Performance Variable Model". Enter "RewardModel1" in the Component Name field and select "Next >".
4. Select "SystemConfiguration1" and click "Finish".
5. Add a new variable called GoalAchieved by entering "GoalAchieved" into the textbox in the top left of the dialog and then pressing "Add Variable:".
6. In the "Rate Rewards" tab, define the reward function to be:

   return SystemConfiguration1->Goal_GainNetworkAccessOnScadaNetwork->Mark();
   

7. Click on the "Apply Changes" button.
8. In the "Time" tab, define the timing type to be an "Instant of Time" with an incremental range. The range should be 0-60 with a step size of 10.
   1. Select the Time Tab
   2. From the Type drop down list select Instant of Time.
   3. From the Time Point definition method select Incremental Range.
   4. In the First time point in series textbox enter 0.0
   5. In the Upper Bound of series textbox enter 60.0
   6. In the Step size in series textbox enter 10.0
   7. Click on the "Apply Changes" button.
9. Save and close the reward model.

You now have a reward model for your ADVISE atomic model that measures the state of the goal.

Defining the Study[edit]

TODO This section may be supplanted in the future.

1. Create a new "Range Study" called "RangeStudy1".
   1. Right click on the ADVISEMetaTutorial project folder and select New.
   2. Select Study in the Component Category in the bottom pane and and select the "Next >" button at the bottom of the dialog.
   3. Select Range Study in the Component Type pane, enter the name RangeStudy1, and click Finish.
   4. Save and close the study by clicking on the File->Save and File->Close menu items.

Since no global variables are defined in our model, we simply need to create an empty study.

Running the Model[edit]

1. Create a new simulation called "Simulation" in the "Solver" folder.
   1. Right click on the ADVISEMetaTutorial project folder and select New.
   2. Select Solver in the Component Category in the bottom pane and and select the "Next >" button at the bottom of the dialog.
   3. Select Simulator in the Component Type pane, enter the name Simulator1.
   4. Select RangeStudy1 as the child, and click Finish.
2. Click on the "Run Simulation" tab and click the "Start Simulation" button.
3. Wait for the simulation to complete; you should now be looking at the "Results" tab.
4. Scroll down to see the mean values of the performance variable at the various time points.
5. Values should be similar to

_________________________________Mean Results_________________________________
Name                  Time               Mean                     Confidence Interval
GoalAchieved          0.0               0.0000000000E00   +/-   0.0000000000E00 
GoalAchieved          10.0              2.5700000000E-02  +/-   3.1016350810E-03  (*)
GoalAchieved          20.0              1.5980000000E-01  +/-   7.1822003087E-03
GoalAchieved          30.0              3.2400000000E-01  +/-   9.1732586744E-03
GoalAchieved          40.0              4.8350000000E-01  +/-   9.7951522163E-03
GoalAchieved          50.0              6.1710000000E-01  +/-   9.5279231056E-03
GoalAchieved          60.0              7.2490000000E-01  +/-   8.7531048801E-03

Click the File->Close menu item.

According to this model there is an approximately 72.5% chance that the Engineer Insider will gain unauthorized access to the SCADA network 60 time units after beginning the attack with this system configuration. We stress that quantitative metrics calculated by ADVISE models are not intended to stand alone. Rather, these metrics are supposed to be compared to metrics that come from similar models. In the next section we will build a model of a similar system.