From e67f812e1865fb8ed630323e6381ccae540a369e Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Wed, 29 Aug 2018 19:17:54 -0400 Subject: [PATCH 001/122] Create ParityPartOne.md --- EducationalAssignments/ParityPartOne.md | 318 ++++++++++++++++++++++++ 1 file changed, 318 insertions(+) create mode 100644 EducationalAssignments/ParityPartOne.md diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md new file mode 100644 index 00000000..7d4aacbf --- /dev/null +++ b/EducationalAssignments/ParityPartOne.md @@ -0,0 +1,318 @@ +# Implement a Defensive Security System + +This assignment will help you understand security mechanisms. You will be +guided through the steps of creating a reference monitor using the security +layer functionality in Repy V2. A reference monitor is an access control +concept that refers to an abstract machine that mediates all access to +objects by subjects. This can be used to allow, deny, or change the +behavior of any set of calls. While not a perfect way of validating your +reference monitor, it is useful to create test cases to see whether your +security layer will work as expected. (The test cases may be turned in as +part of the next assignment.) + +This assignment is intended to reinforce concepts about access control and +reference monitors in a hands-on manner. + + + + + + +## Overview +---- +In this assignment you will create a security layer which prevents writes that +do not maintain even parity for all 8 byte-aligned sequences in a file. +(What this means is described more precisely below.) Parity is often used to +detect errors or tampering for data. + +Your security layer will purely focus on storage of information in a file. +Using the minimal number of read blocks possible, you must determine if an +operation would change the parity. Your security layer must track enough +information about the parity of any 8-byte sequences read so that it need not +read the sequence more often than is necessary. + +All write operations must either complete or be blocked. All writes that would +not cause the parity of a 8-byte sequence to be non-even must be permitted. +Any write that would cause the parity of any 8-byte sequence to be non-even +must be blocked by throwing a RepyParityError exception. + +Note that in some cases there will be an incomplete sequence (e.g., the last +5 bytes of a 13 byte file). Parity is not checked for an incomplete sequence +(it is only checked when the sequence is completed). + +Note that the behavior of other system calls must not be changed in a way +that is visible to the running program. Reading from a file, opening a file, +etc. must appear to operate in the same manner. + +Three design paradigms are at work in this assignment: accuracy, +efficiency, and security. + + * Accuracy: The security layer should stop writeat calls if-and-only-if +they would result in non-even parity. All other actions should be allowed. +For example, if an app tries to read data a file or write data that results +in even parity 8-byte sequences these operations must succeed as per normal +and must not be blocked. All situations that are not described above *must* +match that of the underlying API. + + * Efficiency: The security layer should use a minimum number of resources, +so performance is not compromised. In particular, the security layer may not +read more 8-byte sequences than are necessary. + +SHOULD I GIVE THEM FILESIZE AS A BUILT IN SECURITY LAYER? + + * Security: The attacker should not be able to circumvent the security +layer. Hence, if the attacker can cause a file with a non-even 8-byte sequence +to be written then the security is compromised, for example. + + +### Parity and 8-byte sequences + +For this assignment a file is conceptually broken up into 8-byte sequences. +Every consecutive series of 8 bytes (from the beginning of a file) is its own +8-byte sequence. So, the first 8 bytes (bytes 1-8) are the first sequence, the +next 8 bytes (bytes 9-16) are the second, etc. + +Note that a write may be performed on a non-8-byte-aligned portion of the +file. E.g., bytes 5-17 may be written in a single write. For that write, the +first, second, and third 8 byte sequence are all modified. + +In terms of parity, each byte has a parity based upon its value when calling +chr() in python. Each byte that has a parity divisible by 2 is considered to +be even. An 8-byte sequence is considered to be even if there are an even +number of non-even bytes in the sequence. In other words, if there are 0, +2, 4, 6, or 8 non-even bytes, the sequence is considered to be even. Also, +if a sequence has not been completely written (because it is at the end of a +file), it is always considered to have even parity for the purposes of a +write being allowed or blocked. + + +### Getting Python and RepyV2 + +Please refer to the [SeattleTestbed Build Instructions](../Contributing/BuildInstructions.md#prerequisites) +for details. + +Once you have built RepyV2 into a directory of your choice, change into that +directory. Use the command below in order to run your RepyV2 programs: + +```python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [program].r2py``` + +(Replace `[security_layer].r2py` and `[program].r2py` by the names of the +security layers and program that you want to run.) + +In order to test whether or not these steps worked, please copy and paste +the code found below for the sample security layer and sample attack. + +If you got an error, please go through the troubleshooting section found below. + +### Troubleshooting Repy code +---- +If you can't get Repy files to run, some of the following common errors may +have occurred: + + * using `print` instead of `log`: + +Repy is a subset of Python, but its syntax is slightly different. For +example, Python's `print` statement cannot be used; Repy has `log` for +that. For a full list of acceptable syntax please see +[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md] + + * command line errors: + +**files are missing:** In the above command line call, you must have +`repy.py`, restrictions.default, encasementlib.r2py, the security layer and +the program you want to run in the current working directory. If any or +all of the above files are not in that directory then you will not be able +to run repy files. + + + + +### Tutorials for Repy and Python +---- +Now that you have Repy and Python, you may need a refresher on how to use +them. The following tutorials provide this information. + + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + + + +## Building the security layer +---- +The following program is a basic and incomplete sample code for you to get +an idea about writing security layer. Remember, you have no idea how the +attacker will try to penetrate your security layer, so it is important that +you leave nothing to chance! + + +### A basic (and inadequate) defense + +Time to start coding! Let's inspect a basic security layer. + +``` +""" +This security layer inadequately handles parity for files in RepyV2. + + + +Note: + This security layer uses encasementlib.r2py, restrictions.default, repy.py and Python + Also you need to give it an application to run. + python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py + + """ + +class EvenParityFile(): + def __init__(self,filename,create): + # globals + mycontext['debug'] = False + # local (per object) reference to the underlying file + self.fn = filename + + # make the file + if create: + self.file = openfile(self.fn,create) + + + def writeat(self,data,offset): + + # check the parity of the data written + self.file.writeat(data,offset) + + def readat(self,bytes,offset): + # Read from the file using the sandbox's readat... + return self.file.readat(bytes,offset) + + def close(self): + self.file.close() + + +def parityopenfile(filename, create): + return EvenParityFile(filename,create) + + + + +# The code here sets up type checking and variable hiding for you. You +# should not need to change anything below here. +sec_file_def = {"obj-type":ABFile, + "name":"ABFile", + "writeat":{"type":"func","args":(str,(int,long)),"exceptions":Exception,"return":(int,type(None)),"target":EvenParityFile.writeat}, + "readat":{"type":"func","args":((int,long,type(None)),(int,long)),"exceptions":Exception,"return":str,"target":EvenParityFile.readat}, + "close":{"type":"func","args":None,"exceptions":None,"return":(bool,type(None)),"target":EvenParityFile.close} + } + +CHILD_CONTEXT_DEF["openfile"] = {"type":"objc","args":(str,bool),"exceptions":Exception,"return":sec_file_def,"target":parityopenfile} + +# Execute the user code +secure_dispatch_module() +``` + + + +### Testing your security layer +---- +In this part of the assignment you will pretend to be an attacker. Remember +the attacker's objective is to bypass the parity restrictions or cause +the security layer to act in a disallowed manner. By understanding how the +attacker thinks, you will be able to write better security layers. + +An example of an attack is found below: + +``` +if "testfile.txt.a" in listfiles(): + removefile("testfile.txt.a") +if "testfile.txt.b" in listfiles(): + removefile("testfile.txt.b") +myfile=ABopenfile("testfile.txt",True) #Create an AB file + +# I should get 'SE' when reading an empty file... +assert('SE' == myfile.readat(None,0)) + +# put some valid data in the file. +myfile.writeat("Stest12345E",0) + +# I should still get 'SE' because the file wasn't closed. +assert('SE' == myfile.readat(None,0)) + +#Close the file +myfile.close() + + +``` + +**Note:** All attacks should be written as Repy V2 files, using the .r2py extension. + +#### Choice of File Names +---- +It is important to keep in mind that only lowercase file names are allowed. +So in the above code, specifically: + +``` +# Open a file +myfile=openfile("look.txt",True) +``` + +look.txt is a valid file name, however Look.txt and LOOK.TXT are not. +Examples of other invalid files names are, _look.txt, look/.txt, and +look().txt. Essentially all non-alphanumeric characters are not allowed. + +### Running your security layer +---- +Finally, type the following commands at the terminal to run your security +layer with your attack program + +```python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py ``` + +Make sure you went through the "How to get RepyV2" section! + + +# Notes and Resources +---- + + * For a complete list of syntax in Repyv2 please visit: + * **[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md]** + + * The following link is an excellent source for information about security layers: **[http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf]** + + * **Note:** It is possible to add multiple security layers to Repy, this +may be useful for testing different mitigations separately. This is +done with the following command at the terminal: + +```python repy.py restrictions.default encasementlib.r2py [security_layer1].r2py [security_layer2].r2py [security_layer3].r2py [program].r2py``` + +**Your security layer must produce no output!! ** + + * In repy log replaces print from python. This may be helpful when +testing if Repy installed correctly. + + +# Extra Credit +---- +For extra credit, program that keeps all old versions of files and allows +read from any of them. Writing to any old file creates a new (empty) version +of that file. +Do not submit this code inside your assignment. Submit a separate copy for extra credit. + + + +# What to turn in? +---- + * Turn in a repy file called reference_monitor_[ netid ].r2py with all +letters in lowercase. + +* **Never raise unexpected errors or produce any output.** Your program +must produce no output when run normally. + + * For extra credit turn in a second repy file called extra_credit_[netid].r2py **You must turn in separate files for the normal assignment and extra credit** From 60ddf075403bb36901b6b92955f2f4b300216179 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Wed, 29 Aug 2018 23:06:36 -0400 Subject: [PATCH 002/122] Update ParityPartOne.md --- EducationalAssignments/ParityPartOne.md | 61 +++++++++++++++++-------- 1 file changed, 41 insertions(+), 20 deletions(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index 7d4aacbf..01b4bdd6 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -58,7 +58,7 @@ match that of the underlying API. so performance is not compromised. In particular, the security layer may not read more 8-byte sequences than are necessary. -SHOULD I GIVE THEM FILESIZE AS A BUILT IN SECURITY LAYER? +SHOULD I GIVE THEM FILESIZE AS A BUILT IN A SECURITY LAYER? * Security: The attacker should not be able to circumvent the security layer. Hence, if the attacker can cause a file with a non-even 8-byte sequence @@ -77,7 +77,7 @@ file. E.g., bytes 5-17 may be written in a single write. For that write, the first, second, and third 8 byte sequence are all modified. In terms of parity, each byte has a parity based upon its value when calling -chr() in python. Each byte that has a parity divisible by 2 is considered to +ord() in python. Each byte that has a parity divisible by 2 is considered to be even. An 8-byte sequence is considered to be even if there are an even number of non-even bytes in the sequence. In other words, if there are 0, 2, 4, 6, or 8 non-even bytes, the sequence is considered to be even. Also, @@ -180,15 +180,32 @@ class EvenParityFile(): # local (per object) reference to the underlying file self.fn = filename - # make the file + # create the file on disk if create: self.file = openfile(self.fn,create) def writeat(self,data,offset): - + # check the parity of the data written - self.file.writeat(data,offset) + # NOTE: This is wrong in many ways!!!! + thisdata = data + while thisdata: + eightbytesequence = thisdata[:8] + thisdata = thisdata[8:] + even = True + for thisbyte in eightbytesequence: + # for each byte, if it is odd, flip even to be the opposite + if ord(thisbyte) % 2: + even = not even + + # actually call write, if we are supposed to... + if even: + self.file.writeat(eightbytesequence,offset) + # ...or error out. + else: + raise RepyParityError("Non-even parity write to file") + def readat(self,bytes,offset): # Read from the file using the sandbox's readat... @@ -206,8 +223,8 @@ def parityopenfile(filename, create): # The code here sets up type checking and variable hiding for you. You # should not need to change anything below here. -sec_file_def = {"obj-type":ABFile, - "name":"ABFile", +sec_file_def = {"obj-type":EvenParityFile, + "name":"EvenParityFile", "writeat":{"type":"func","args":(str,(int,long)),"exceptions":Exception,"return":(int,type(None)),"target":EvenParityFile.writeat}, "readat":{"type":"func","args":((int,long,type(None)),(int,long)),"exceptions":Exception,"return":str,"target":EvenParityFile.readat}, "close":{"type":"func","args":None,"exceptions":None,"return":(bool,type(None)),"target":EvenParityFile.close} @@ -228,24 +245,28 @@ the attacker's objective is to bypass the parity restrictions or cause the security layer to act in a disallowed manner. By understanding how the attacker thinks, you will be able to write better security layers. -An example of an attack is found below: +An example of a test / attack is found below: ``` -if "testfile.txt.a" in listfiles(): - removefile("testfile.txt.a") -if "testfile.txt.b" in listfiles(): - removefile("testfile.txt.b") -myfile=ABopenfile("testfile.txt",True) #Create an AB file +if "testfile.txt" in listfiles(): + removefile("testfile.txt") -# I should get 'SE' when reading an empty file... -assert('SE' == myfile.readat(None,0)) +myfile=openfile("testfile.txt",True) #Create a parity file # put some valid data in the file. -myfile.writeat("Stest12345E",0) - -# I should still get 'SE' because the file wasn't closed. -assert('SE' == myfile.readat(None,0)) - +myfile.writeat("AA",0) + +# I should be able to read it out. +assert('AA' == myfile.readat(None,0)) + +# However, this write should fail... +try: + myfile.writeat("BCBCBC",2) +except RepyParityError: + pass # should happen +else: + log("should have been an error instead!") + #Close the file myfile.close() From 414e8bf41786a745ca722e319aabc4d96bd66ca2 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 4 Sep 2018 16:47:28 -0400 Subject: [PATCH 003/122] Update ParityPartOne.md --- EducationalAssignments/ParityPartOne.md | 6 ++---- 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index 01b4bdd6..2544db40 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -179,10 +179,8 @@ class EvenParityFile(): mycontext['debug'] = False # local (per object) reference to the underlying file self.fn = filename - - # create the file on disk - if create: - self.file = openfile(self.fn,create) + + self.file = openfile(self.fn,create) def writeat(self,data,offset): From 64dea2f7b698bafad75bb6ca201ad26f11faa0b9 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 4 Sep 2018 16:57:07 -0400 Subject: [PATCH 004/122] Adding IFL security layer. --- EducationalAssignments/ParityPartOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index 2544db40..6cc9b5cd 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -94,7 +94,7 @@ for details. Once you have built RepyV2 into a directory of your choice, change into that directory. Use the command below in order to run your RepyV2 programs: -```python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [program].r2py``` +```python repy.py restrictions.default encasementlib.r2py initialfilelength.r2py [security_layer].r2py [program].r2py``` (Replace `[security_layer].r2py` and `[program].r2py` by the names of the security layers and program that you want to run.) From 55e2b2c666e2b58dd31f7004048369ec6eba962f Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Thu, 13 Sep 2018 17:48:20 -0400 Subject: [PATCH 005/122] Don't need file size --- EducationalAssignments/ParityPartOne.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index 6cc9b5cd..f3a8c41d 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -58,8 +58,6 @@ match that of the underlying API. so performance is not compromised. In particular, the security layer may not read more 8-byte sequences than are necessary. -SHOULD I GIVE THEM FILESIZE AS A BUILT IN A SECURITY LAYER? - * Security: The attacker should not be able to circumvent the security layer. Hence, if the attacker can cause a file with a non-even 8-byte sequence to be written then the security is compromised, for example. From 2d1ae257e7b3bbb0c90636971e6557b20ce128ed Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Thu, 13 Sep 2018 17:51:25 -0400 Subject: [PATCH 006/122] Update ParityPartOne.md --- EducationalAssignments/ParityPartOne.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index f3a8c41d..438e80ed 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -38,7 +38,7 @@ must be blocked by throwing a RepyParityError exception. Note that in some cases there will be an incomplete sequence (e.g., the last 5 bytes of a 13 byte file). Parity is not checked for an incomplete sequence -(it is only checked when the sequence is completed). +(it is only checked when the sequence is completed). Note that the behavior of other system calls must not be changed in a way that is visible to the running program. Reading from a file, opening a file, @@ -56,7 +56,8 @@ match that of the underlying API. * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. In particular, the security layer may not -read more 8-byte sequences than are necessary. +read more 8-byte sequences than are necessary. Hint: it is *always* possible +to read two or fewer 8 byte blocks per writeat(). * Security: The attacker should not be able to circumvent the security layer. Hence, if the attacker can cause a file with a non-even 8-byte sequence From f901156adfb05aab4a582e526be91acc05d58fea Mon Sep 17 00:00:00 2001 From: SanjayTr <35617078+SanjayTr@users.noreply.github.com> Date: Sun, 7 Oct 2018 20:51:57 -0400 Subject: [PATCH 007/122] Updating the instructions removed 'initialfilelength.r2py' from the command in line 96 removed information/instructions regarding the extra credit. Instructions on how to run the Repy program is mentioned twice at line 96 and also at 291. I have retained both. --- EducationalAssignments/ParityPartOne.md | 14 +------------- 1 file changed, 1 insertion(+), 13 deletions(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index 438e80ed..a00f30c4 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -93,7 +93,7 @@ for details. Once you have built RepyV2 into a directory of your choice, change into that directory. Use the command below in order to run your RepyV2 programs: -```python repy.py restrictions.default encasementlib.r2py initialfilelength.r2py [security_layer].r2py [program].r2py``` +```python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [program].r2py``` (Replace `[security_layer].r2py` and `[program].r2py` by the names of the security layers and program that you want to run.) @@ -315,16 +315,6 @@ done with the following command at the terminal: * In repy log replaces print from python. This may be helpful when testing if Repy installed correctly. - -# Extra Credit ----- -For extra credit, program that keeps all old versions of files and allows -read from any of them. Writing to any old file creates a new (empty) version -of that file. -Do not submit this code inside your assignment. Submit a separate copy for extra credit. - - - # What to turn in? ---- * Turn in a repy file called reference_monitor_[ netid ].r2py with all @@ -332,5 +322,3 @@ letters in lowercase. * **Never raise unexpected errors or produce any output.** Your program must produce no output when run normally. - - * For extra credit turn in a second repy file called extra_credit_[netid].r2py **You must turn in separate files for the normal assignment and extra credit** From 16478a06e78083a4c36f04d503210d665e5aeb8f Mon Sep 17 00:00:00 2001 From: aaaaalbert Date: Tue, 9 Oct 2018 15:17:40 +0200 Subject: [PATCH 008/122] Fix code style in Parity assignment Also fix capitalization / rendering of RepyV2 and Python, and improve the Markdown throughout. --- EducationalAssignments/ParityPartOne.md | 47 +++++++++++++------------ 1 file changed, 24 insertions(+), 23 deletions(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index a00f30c4..75dff59e 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -2,7 +2,7 @@ This assignment will help you understand security mechanisms. You will be guided through the steps of creating a reference monitor using the security -layer functionality in Repy V2. A reference monitor is an access control +layer functionality in RepyV2. A reference monitor is an access control concept that refers to an abstract machine that mediates all access to objects by subjects. This can be used to allow, deny, or change the behavior of any set of calls. While not a perfect way of validating your @@ -76,7 +76,7 @@ file. E.g., bytes 5-17 may be written in a single write. For that write, the first, second, and third 8 byte sequence are all modified. In terms of parity, each byte has a parity based upon its value when calling -ord() in python. Each byte that has a parity divisible by 2 is considered to +`ord()` in Python. Each byte that has a parity divisible by 2 is considered to be even. An 8-byte sequence is considered to be even if there are an even number of non-even bytes in the sequence. In other words, if there are 0, 2, 4, 6, or 8 non-even bytes, the sequence is considered to be even. Also, @@ -173,13 +173,13 @@ Note: """ class EvenParityFile(): - def __init__(self,filename,create): + def __init__(self, filename, create): # globals mycontext['debug'] = False # local (per object) reference to the underlying file self.fn = filename - self.file = openfile(self.fn,create) + self.file = openfile(self.fn, create) def writeat(self,data,offset): @@ -198,22 +198,22 @@ class EvenParityFile(): # actually call write, if we are supposed to... if even: - self.file.writeat(eightbytesequence,offset) + self.file.writeat(eightbytesequence, offset) # ...or error out. else: raise RepyParityError("Non-even parity write to file") - def readat(self,bytes,offset): + def readat(self, bytes, offset): # Read from the file using the sandbox's readat... - return self.file.readat(bytes,offset) + return self.file.readat(bytes, offset) def close(self): self.file.close() def parityopenfile(filename, create): - return EvenParityFile(filename,create) + return EvenParityFile(filename, create) @@ -250,27 +250,27 @@ if "testfile.txt" in listfiles(): myfile=openfile("testfile.txt",True) #Create a parity file -# put some valid data in the file. -myfile.writeat("AA",0) +# Put some valid data in the file. +myfile.writeat("AA", 0) # I should be able to read it out. -assert('AA' == myfile.readat(None,0)) +assert('AA' == myfile.readat(None, 0)) # However, this write should fail... try: - myfile.writeat("BCBCBC",2) + myfile.writeat("BCBCBC", 2) except RepyParityError: pass # should happen else: log("should have been an error instead!") -#Close the file +# Close the file myfile.close() ``` -**Note:** All attacks should be written as Repy V2 files, using the .r2py extension. +**Note:** All attacks should be written as RepyV2 files, using the `.r2py` extension. #### Choice of File Names ---- @@ -279,12 +279,12 @@ So in the above code, specifically: ``` # Open a file -myfile=openfile("look.txt",True) +myfile=openfile("look.txt", True) ``` -look.txt is a valid file name, however Look.txt and LOOK.TXT are not. -Examples of other invalid files names are, _look.txt, look/.txt, and -look().txt. Essentially all non-alphanumeric characters are not allowed. +`look.txt` is a valid file name, however `Look.txt` and `LOOK.TXT` are not. +Examples of other invalid files names are `.look.txt`, `look/.txt`, and +`look().txt`. Essentially all non-alphanumeric characters are not allowed. ### Running your security layer ---- @@ -299,10 +299,11 @@ Make sure you went through the "How to get RepyV2" section! # Notes and Resources ---- - * For a complete list of syntax in Repyv2 please visit: - * **[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md]** + * For a complete list of syntax in Repyv2 please visit + the **[RepyV2 API documentation](https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md)** - * The following link is an excellent source for information about security layers: **[http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf]** + * **[This paper](http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf)** + is an excellent source for information about security layers * **Note:** It is possible to add multiple security layers to Repy, this may be useful for testing different mitigations separately. This is @@ -310,9 +311,9 @@ done with the following command at the terminal: ```python repy.py restrictions.default encasementlib.r2py [security_layer1].r2py [security_layer2].r2py [security_layer3].r2py [program].r2py``` -**Your security layer must produce no output!! ** +**Your security layer must produce no output!!** - * In repy log replaces print from python. This may be helpful when + * In RepyV2, `log` replaces `print` from Python. This may be helpful when testing if Repy installed correctly. # What to turn in? From 8362efef33a5623ad6f73d4e703aed85a1869d9e Mon Sep 17 00:00:00 2001 From: SanjayTr <35617078+SanjayTr@users.noreply.github.com> Date: Thu, 18 Oct 2018 17:52:23 -0400 Subject: [PATCH 009/122] 2 --- EducationalAssignments/ParityPartTwo(2).md | 217 +++++++++++++++++++++ 1 file changed, 217 insertions(+) create mode 100644 EducationalAssignments/ParityPartTwo(2).md diff --git a/EducationalAssignments/ParityPartTwo(2).md b/EducationalAssignments/ParityPartTwo(2).md new file mode 100644 index 00000000..14cd4f66 --- /dev/null +++ b/EducationalAssignments/ParityPartTwo(2).md @@ -0,0 +1,217 @@ + +# Security layer testing and penetration + +In this assignment you will learn how to attack a reference monitor. The reference monitor you will be testing uses the security layer framework (encasement library, etc.) for the Seattle testbed. It is possible to do this assignment separately, but it is recommended that this assignment be completed after [Part One](https://github.com/SeattleTestbed/docs/blob/JustinCappos-ParityAssignment/EducationalAssignments/ParityPartOne.md). Either way you should already have a working security layer or access to one. Testing the security layer is done by running a series of test cases that an adversary may use to circumvent your system. This assignment is intended to prepare you for thinking about security paradigms in a functional way. The ideas of information, security and privacy have been embedded into the steps of this assignment. + + + + +## Overview +---- +In this assignment you are a tester. You have been sent a bunch of reference monitors that need testing before they are deployed. Your job will be to ensure an attacker cannot circumvent these security layers. In order to do this, you will attempt to read, write, and append data without having the proper missions enabled. If you are able to do so, then the security layer is not secure. The future of the system depends on your ability to test code thoroughly! + +Three design paradigms are at work in this assignment: accuracy, efficiency, and security. + + * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. + + * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. + + * Security: The attacker should not be able to circumvent the security layer. + +Within the context of this assignment these design paradigms translate to: + + * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. For example, if an app tries to read data or write valid data to a file, this must succeed as per normal and must not be blocked. All situations that are not described above must match that of the underlying API. + + * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. For example, reading more blocks of existing data from the file, than necessary, would be forbidden. + + * Security: The attacker should not be able to circumvent the security layer. Hence, if the attacker can cause an invalid write to a file, then the security is compromised, for example. + +You will submit a zip file containing all of the tests you have created. You will gain points for every student's reference monitor you find a flaw in. It is good if multiple tests of yours break a student's reference monitor, but you gain the same number of tests whether one or more tests break the layer. + + + +## Prerequisites + +This assignment assumes you have both the latest Python 2.7 and RepyV2 +installed on your computer. Please refer to the [SeattleTestbed Build Instructions](../Contributing/BuildInstructions.md#prerequisites) +for information on how to get them. + + +### Helpful links +---- +The following links will aid students in becoming comfortable with Python, Repy and seattle: + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + + +## Testing security layers +---- +### Hypothesis, test case, counter example + +The goal of a good tester is to test hypotheses. A hypothesis is just a scientific way of asking a question. The hypothesis of this assignment is "This security layer is well designed." The questions you will ask when running your test cases will always be the same + + * "Is this reference monitor secure?" + + * "Does this reference monitor hamper performance?" + + * "Does this reference monitor prevent actions that should be allowed?" + +Notice that these questions are parallels of the security paradigms: security, efficiency and accuracy, respectively. + +If we can find a case where the hypothesis is false, then the security layer is not secure. Such a case is referred to as a counter example. Hence all test cases should be designed to test for these three types of flaws. + + + +### Examples of tests +Test cases are briefly described at [https://github.com/SeattleTestbed/docs/blob/JustinCappos-ParityAssignment/EducationalAssignments/ParityPartOne.md] and [wiki:RepyV2SecurityLayers]. Below is another example of a test case you may want to consider. This test case gives the right 'style' for all your test cases, but lacks in the number of test cases. A good attack will include many test cases. +#### Test case 1: + +``` +# Valid Write Operation on an empty file + +# Clean up of existing file +if "testfile.txt" in listfiles(): + removefile("testfile.txt") + +# Open File Function Call +myfile=openfile("testfile.txt",True) #Create a file + +try: + # write valid data onto the file. + myfile.writeat("ABBA",0) + # read from the file to see if the write was successful. + assert('ABBA' == myfile.readat(4,0)) + # Close the file: + myfile.close() +except: + myfile.close() + # Error Handle or Failure Condition + log("Valid data not written!") + +``` +#### Code analysis +It is important to keep in mind that only lowercase file names are allowed. So in the above code, specifically: + +``` + +# Open a file +myfile=openfile("testfile.txt",True) + +``` +testfile.txt is a valid file name, however Testfile.txt is not. Examples of other invalid files names are, testfile@.txt, testfile/.txt, and testfile().txt. Essentially all non-alphanumeric characters are not allowed. + +In this case we are verifying the security of the reference monitor. This code attempts to write valid data on an empty file and checks if the reference monitor allowed the write. First the existing testfile is removed and a new testfile is created. Next, valid data is written onto the file and then we check if the reference monitor behaved as expected, that is, if it allowed the write. `myfile.readat(4,0)` tries to read from the file which has been created. The 0 refers to an offset of zero and 4 refers to number of characters being read. The assert call would look to validate the condition mentioned within its call `Check for valid data string "ABBA"`and pass if the statement made is true or raise exception if the statement made is false which can be caught using exception handlers. Then finally: statement will always run, closing the file. + +#### Test case 2: + +``` +# Valid WRITE on a non-empty file + +# New File Operation, Clean up of existing file +if "testfile.txt" in listfiles(): + removefile("testfile.txt") + +# Open File Function Call +myfile=openfile("testfile.txt",True) #Create a file + +# Write valid data to the file +myfile.writeat("AAAAAAA",0) +# Write data over existing data in the file +myfile.writeat("A",7) + +# Read the file to check the contents +try: + assert('AAAAAAAA' == myfile.readat(8,0)) + #Close the file + myfile.close() +except: + #Close the file + myfile.close() + log("Valid Data write to a file is unsuccessfull!") + +``` +#### Code analysis + +In this case we are verifying the accuracy of the reference monitor. This code attempts to write `"A"` to the file that already has `"AAAAAAA"` and verify the contents. First the file is opened using `myfile=openfile("testfile.txt",True)`. Next `myfile.writeat("A",7)` tries to write `"A"` to the file at offset 7. Then `myfile.readat(8,0)` tries to read the contents of the file. The 8 refers to number of characters to read and 0 refers to the offset 0 in the file. If the security layer fails the test then the assert call raises exception to be caught by except statement to show the error. The final statement which will always run, closing the file. + +If this case produces anything other than "No Output", then this layer fails the accuracy design paradigm. The security layer should allow valid writes onto the file. + +#### More information on: Try, Except, Else, Finally +The try, except, else and finally statements are part of **exception handling**. For more information on exception handling please visit: + + * [http://docs.python.org/tutorial/errors.html] + * [http://wiki.python.org/moin/HandlingExceptions] + * [http://www.tutorialspoint.com/python/python_exceptions.htm] + +### Hints and Ideas for testing + +When writing your own tests it is important to test for a complete set of possible penetrations. Keep in mind, it only takes one test case to break through a security layer. Some of the things you may want to test for include: + + * threading + * writing to multiple files???? + * multiple writes + +And more! Remember a good security layer can't be broken by anyone! Which is all part of the fun! It's about solving a puzzle. First you make the puzzle - write the security layer, then you solve the puzzle - try to bypass it. If your puzzle is "good enough", no one will be able to break it, no matter what. + + +## Notes and Resources +---- + + * The following link is an excellent source for information about security layers: http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf + + * [repy_v2/benchmarking-support/allnoopsec.py](https://seattle.poly.edu/browser/seattle/branches/repy_v2/benchmarking-support/allnoopsec.py) is an empty security layer that doesn't perform any operations. + + * [repy_v2/benchmarking-support/all-logsec.py](https://seattle.poly.edu/browser/seattle/branches/repy_v2/benchmarking-support/all-logsec.py) is security layer that performs logging functions. + + * In repy 'log' replaces 'print' from python. Many students find this to be a stumbling block. + + * Note that you should not assume that any files exist in your directory. You should create any files (e.g., testfile.txt) yourself in your test program. + + + + +## How to run your tests on many reference monitors +---- + +Create a directory that the security layers will write their files into. You need to run repy with only access to this directory. You can write a test program that does `log(str(listfiles()))` to see if you are in the right place. + +Have all the reference monitors to test and the test cases inside the same directory where the repy.py file exists. + +* In the bash shell on Mac and Linux: +``` +for referencemonitor in reference_monitor_*; do for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` +* In the Command Prompt on Windows: +``` +FOR %r IN (reference_monitor_*) DO @FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a +``` +* In PowerShell on Windows: +``` +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the output from each program. Make sure that you replace `` with your NetID. + +If you want to spot the referencemonitor that failed during the test run, add echo the name of each referencemonitor before the inner loop, like so: + +* In the bash shell on Mac and Linux: +``` +for referencemonitor in reference_monitor_*; do echo $referencemonitor under test; for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` +* In the Command Prompt on Windows: +``` +FOR %r IN (reference_monitor_*) DO @(ECHO %r under test & FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a) +``` +* In PowerShell on Windows: +``` +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { Write-Host $referencemonitor.Name; foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the name of each reference monitor before it starts executing the testcases against it. + + +## What to turn in? +---- + + * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must start with your netid in lowercase. For example: abc123_securitytest1.r2py abc123_goodaccuracytest.r2py are both valid names. From c959fd81c8ee059e02ff0f175e32ff8062d95dfb Mon Sep 17 00:00:00 2001 From: SanjayTr <35617078+SanjayTr@users.noreply.github.com> Date: Thu, 18 Oct 2018 21:08:06 -0400 Subject: [PATCH 010/122] Reverting ABStoragePartTwo --- .../{ParityPartTwo.md => ABStoragePartTwo.md} | 76 +++++++++++-------- 1 file changed, 45 insertions(+), 31 deletions(-) rename EducationalAssignments/{ParityPartTwo.md => ABStoragePartTwo.md} (65%) diff --git a/EducationalAssignments/ParityPartTwo.md b/EducationalAssignments/ABStoragePartTwo.md similarity index 65% rename from EducationalAssignments/ParityPartTwo.md rename to EducationalAssignments/ABStoragePartTwo.md index 14cd4f66..1c0e3298 100644 --- a/EducationalAssignments/ParityPartTwo.md +++ b/EducationalAssignments/ABStoragePartTwo.md @@ -1,7 +1,7 @@ # Security layer testing and penetration -In this assignment you will learn how to attack a reference monitor. The reference monitor you will be testing uses the security layer framework (encasement library, etc.) for the Seattle testbed. It is possible to do this assignment separately, but it is recommended that this assignment be completed after [Part One](https://github.com/SeattleTestbed/docs/blob/JustinCappos-ParityAssignment/EducationalAssignments/ParityPartOne.md). Either way you should already have a working security layer or access to one. Testing the security layer is done by running a series of test cases that an adversary may use to circumvent your system. This assignment is intended to prepare you for thinking about security paradigms in a functional way. The ideas of information, security and privacy have been embedded into the steps of this assignment. +In this assignment you will learn how to attack a reference monitor. The reference monitor you will be testing uses the security layer framework (encasement library, etc.) for the Seattle testbed. It is possible to do this assignment separately, but it is recommended that this assignment be completed after [Part One](ABStoragePartOne.md). Either way you should already have a working security layer or access to one. Testing the security layer is done by running a series of test cases that an adversary may use to circumvent your system. This assignment is intended to prepare you for thinking about security paradigms in a functional way. The ideas of information, security and privacy have been embedded into the steps of this assignment. @@ -20,13 +20,13 @@ Three design paradigms are at work in this assignment: accuracy, efficiency, and Within the context of this assignment these design paradigms translate to: - * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. For example, if an app tries to read data or write valid data to a file, this must succeed as per normal and must not be blocked. All situations that are not described above must match that of the underlying API. + * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. For example, if an app tries to read data from a valid file, this must succeed as per normal and must not be blocked. All situations that are not described above must match that of the underlying API. - * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. For example, reading more blocks of existing data from the file, than necessary, would be forbidden. + * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. For example, keeping a complete copy of every file on disk in memory would be forbidden. - * Security: The attacker should not be able to circumvent the security layer. Hence, if the attacker can cause an invalid write to a file, then the security is compromised, for example. + * Security: The attacker should not be able to circumvent the security layer. Hence, if the attacker can cause an invalid file to be read or can write to a valid file, then the security is compromised, for example. -You will submit a zip file containing all of the tests you have created. You will gain points for every student's reference monitor you find a flaw in. It is good if multiple tests of yours break a student's reference monitor, but you gain the same number of tests whether one or more tests break the layer. +You will submit a zip file containing all of the tests you have created. You will gain points for every student's reference monitor you find a flaw in. It is good if multiple tests of yours break a student's reference monitor, but you gain the same number of tests whether one or more tests break the layer. @@ -64,30 +64,30 @@ If we can find a case where the hypothesis is false, then the security layer is ### Examples of tests -Test cases are briefly described at [https://github.com/SeattleTestbed/docs/blob/JustinCappos-ParityAssignment/EducationalAssignments/ParityPartOne.md] and [wiki:RepyV2SecurityLayers]. Below is another example of a test case you may want to consider. This test case gives the right 'style' for all your test cases, but lacks in the number of test cases. A good attack will include many test cases. +Test cases are briefly described at [github: SeattleTestbed/docs/blob/master/EducationalAssignments/ABStoragePartOne.md] and [wiki:RepyV2SecurityLayers]. Below is another example of a test case you may want to consider. This test case gives the right 'style' for your all your test cases, but lacks in the number of test cases. A good attack will include many test cases. #### Test case 1: ``` -# Valid Write Operation on an empty file +# New File Operation # Clean up of existing file -if "testfile.txt" in listfiles(): - removefile("testfile.txt") +if "testfile.txt.a" in listfiles(): + removefile("testfile.txt.a") +if "testfile.txt.b" in listfiles(): + removefile("testfile.txt.b") # Open File Function Call -myfile=openfile("testfile.txt",True) #Create a file +myfile=ABopenfile("testfile.txt",True) #Create an AB file try: - # write valid data onto the file. - myfile.writeat("ABBA",0) - # read from the file to see if the write was successful. - assert('ABBA' == myfile.readat(4,0)) + # Empty/New File should have contents 'SE' satisfying the requirement + assert('SE' == myfile.readat(2,0)) # Close the file: myfile.close() except: myfile.close() # Error Handle or Failure Condition - log("Valid data not written!") + log("Empty file is not handled properly!") ``` #### Code analysis @@ -96,33 +96,41 @@ It is important to keep in mind that only lowercase file names are allowed. So ``` # Open a file -myfile=openfile("testfile.txt",True) +myfile=ABopenfile("testfile.txt",True) ``` -testfile.txt is a valid file name, however Testfile.txt is not. Examples of other invalid files names are, testfile@.txt, testfile/.txt, and testfile().txt. Essentially all non-alphanumeric characters are not allowed. +testfile.txt is a valid file name, however Testfile.txt is not. Examples of other invalid files names are, testfile@.txt, testfile/.txt, and testfile().txt. Essentially all non-alphanumeric characters are not allowed. -In this case we are verifying the security of the reference monitor. This code attempts to write valid data on an empty file and checks if the reference monitor allowed the write. First the existing testfile is removed and a new testfile is created. Next, valid data is written onto the file and then we check if the reference monitor behaved as expected, that is, if it allowed the write. `myfile.readat(4,0)` tries to read from the file which has been created. The 0 refers to an offset of zero and 4 refers to number of characters being read. The assert call would look to validate the condition mentioned within its call `Check for valid data string "ABBA"`and pass if the statement made is true or raise exception if the statement made is false which can be caught using exception handlers. Then finally: statement will always run, closing the file. +In this case we are verifying the security of the reference monitor. This code attempts to check the contents of a new valid file created through reference monitor. First the file is opened using ABopenfile function `myfile=ABopenfile("testfile.txt",True)`. Next `myfile.readat(2,0)` tries to read from the file which has been created. The 0 refers to an offset of zero and 2 refers to number of characters being read. The assert call would look to validate the condition mentioned within its call `Check for valid empty file string "SE"`and pass if the statement made is true or raise exception if the statement made is false which can be caught using exception handlers. Then finally: statement will always run, closing the file. #### Test case 2: ``` -# Valid WRITE on a non-empty file +# WRITE OPERATION +# New File Operation -# New File Operation, Clean up of existing file -if "testfile.txt" in listfiles(): - removefile("testfile.txt") +# Clean up of existing file +if "testfile.txt.a" in listfiles(): + removefile("testfile.txt.a") +if "testfile.txt.b" in listfiles(): + removefile("testfile.txt.b") # Open File Function Call -myfile=openfile("testfile.txt",True) #Create a file +myfile=ABopenfile("testfile.txt",True) #Create an AB file # Write valid data to the file -myfile.writeat("AAAAAAA",0) -# Write data over existing data in the file -myfile.writeat("A",7) +myfile.writeat("StestE",0) + +#Close the file +myfile.close() + +# READ OPERATION +# Reopen file again to read +myfile=ABopenfile("testfile.txt",True) # Read the file to check the contents try: - assert('AAAAAAAA' == myfile.readat(8,0)) + assert('StestE' == myfile.readat(6,0)) #Close the file myfile.close() except: @@ -133,9 +141,9 @@ except: ``` #### Code analysis -In this case we are verifying the accuracy of the reference monitor. This code attempts to write `"A"` to the file that already has `"AAAAAAA"` and verify the contents. First the file is opened using `myfile=openfile("testfile.txt",True)`. Next `myfile.writeat("A",7)` tries to write `"A"` to the file at offset 7. Then `myfile.readat(8,0)` tries to read the contents of the file. The 8 refers to number of characters to read and 0 refers to the offset 0 in the file. If the security layer fails the test then the assert call raises exception to be caught by except statement to show the error. The final statement which will always run, closing the file. +In this case we are verifying the accuracy of the reference monitor. This code attempts to write `"StestE"` to the file and verify the contents. We assume that valid data is written in the file from the zero offset. First the file is opened using `myfile=ABopenfile("testfile.txt",True)`. Next `myfile.writeat("StestE",0)` tries to write `"StestE"` to the file. Then `myfile.readat(6,0)` tries to read the contents of the file. The 6 refers to an byte size of 6 characters and 0 refers to the offset 0 in the file. If the security layer fails the test then the assert call raises exception to be caught by except statement to show the error. The final statement which will always run, closing the file. -If this case produces anything other than "No Output", then this layer fails the accuracy design paradigm. The security layer should allow valid writes onto the file. +If this case produces anything other than "No Output", then this layer fails the accuracy design paradigm. The security layer should not stop a file from being read or written. #### More information on: Try, Except, Else, Finally The try, except, else and finally statements are part of **exception handling**. For more information on exception handling please visit: @@ -149,8 +157,7 @@ The try, except, else and finally statements are part of **exception handling**. When writing your own tests it is important to test for a complete set of possible penetrations. Keep in mind, it only takes one test case to break through a security layer. Some of the things you may want to test for include: * threading - * writing to multiple files???? - * multiple writes + * writing to multiple files And more! Remember a good security layer can't be broken by anyone! Which is all part of the fun! It's about solving a puzzle. First you make the puzzle - write the security layer, then you solve the puzzle - try to bypass it. If your puzzle is "good enough", no one will be able to break it, no matter what. @@ -171,6 +178,12 @@ And more! Remember a good security layer can't be broken by anyone! Which is a +## Extra Credit +---- +Find bugs in the extra credit reference monitors given the altered threat model. You should include more test cases in the extra credit! + + + ## How to run your tests on many reference monitors ---- @@ -215,3 +228,4 @@ This will print out the name of each reference monitor before it starts executin ---- * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must start with your netid in lowercase. For example: abc123_securitytest1.r2py abc123_goodaccuracytest.r2py are both valid names. + * Optionally turn in the test cases used to attack the extra credit reference monitors in a zip file. Note that in this case, you can expect that your code is run more than once. In the name of the file, say if it needs to be run multiple times. For example: abc123_run_twice_metadata_removal.r2py abc123_run_once_threading_hack.r2py. From 342921bf16061d2038cf9aa31f5c0bc9da251c4a Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Fri, 19 Oct 2018 09:27:44 -0400 Subject: [PATCH 011/122] n --- EducationalAssignments/{ParityPartTwo(2).md => ParityPartTwo.md} | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename EducationalAssignments/{ParityPartTwo(2).md => ParityPartTwo.md} (100%) diff --git a/EducationalAssignments/ParityPartTwo(2).md b/EducationalAssignments/ParityPartTwo.md similarity index 100% rename from EducationalAssignments/ParityPartTwo(2).md rename to EducationalAssignments/ParityPartTwo.md From e8f3663a1048c05b146cb5170903405d68f821e8 Mon Sep 17 00:00:00 2001 From: SanjayTr <35617078+SanjayTr@users.noreply.github.com> Date: Fri, 19 Oct 2018 09:37:56 -0400 Subject: [PATCH 012/122] line 152, removing '?' --- EducationalAssignments/ParityPartTwo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/ParityPartTwo.md b/EducationalAssignments/ParityPartTwo.md index 14cd4f66..a3971b00 100644 --- a/EducationalAssignments/ParityPartTwo.md +++ b/EducationalAssignments/ParityPartTwo.md @@ -149,7 +149,7 @@ The try, except, else and finally statements are part of **exception handling**. When writing your own tests it is important to test for a complete set of possible penetrations. Keep in mind, it only takes one test case to break through a security layer. Some of the things you may want to test for include: * threading - * writing to multiple files???? + * writing to multiple files * multiple writes And more! Remember a good security layer can't be broken by anyone! Which is all part of the fun! It's about solving a puzzle. First you make the puzzle - write the security layer, then you solve the puzzle - try to bypass it. If your puzzle is "good enough", no one will be able to break it, no matter what. From a121f19573579ea8043d02157f7d8f555617deb4 Mon Sep 17 00:00:00 2001 From: SanjayTr <35617078+SanjayTr@users.noreply.github.com> Date: Sat, 20 Oct 2018 17:22:30 -0400 Subject: [PATCH 013/122] Adding RepyParityError --- EducationalAssignments/ParityPartOne.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index 75dff59e..436a9d0d 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -172,6 +172,9 @@ Note: """ +class RepyParityError(Exception): + pass + class EvenParityFile(): def __init__(self, filename, create): # globals @@ -228,7 +231,7 @@ sec_file_def = {"obj-type":EvenParityFile, } CHILD_CONTEXT_DEF["openfile"] = {"type":"objc","args":(str,bool),"exceptions":Exception,"return":sec_file_def,"target":parityopenfile} - +CHILD_CONTEXT_DEF["RepyParityError"] = {"type":"any","target":RepyParityError} # Execute the user code secure_dispatch_module() ``` From ddd90aaebbed2b31cd10adc0223ab7f246300d32 Mon Sep 17 00:00:00 2001 From: SanjayTr <35617078+SanjayTr@users.noreply.github.com> Date: Wed, 24 Oct 2018 20:43:51 -0400 Subject: [PATCH 014/122] Updating naming convention Original line was confusing and it was probably the reason why few students included '[' and ']' in their file names. --- EducationalAssignments/ParityPartThree.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/ParityPartThree.md b/EducationalAssignments/ParityPartThree.md index d0cf3409..a7f71c50 100644 --- a/EducationalAssignments/ParityPartThree.md +++ b/EducationalAssignments/ParityPartThree.md @@ -52,7 +52,7 @@ Let's analyze one of the bugs in this implementation. According to the specific ## What to turn in? ---- - * Turn in the reference monitor that you have fixed. The name of the reference monitor should be in the form of reference_monitor_[poly_username].r2py + * Turn in the reference monitor that you have fixed. The name of the reference monitor should be in the form of reference_monitor_netId.r2py e.g. reference_monitor_jcappos.r2py All letters must be lowercase. * In addition to your reference monitor, submit a one-page PDF document, in which you discuss the different classes of bugs you had in your reference monitors and why. From 47685b448059f6d6afb8c0aa0869c58b0bfcdfe1 Mon Sep 17 00:00:00 2001 From: SanjayTr <35617078+SanjayTr@users.noreply.github.com> Date: Thu, 25 Oct 2018 00:19:31 -0400 Subject: [PATCH 015/122] Updating naming convention --- EducationalAssignments/ParityPartOne.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index 436a9d0d..99d732ab 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -321,8 +321,8 @@ testing if Repy installed correctly. # What to turn in? ---- - * Turn in a repy file called reference_monitor_[ netid ].r2py with all -letters in lowercase. + * Turn in a repy file called `reference_monitor_netid.r2py` with all +letters in lowercase. For example, if your netId is `jc123`, your reference monitor should be named `reference_monitor_jc123.r2py` * **Never raise unexpected errors or produce any output.** Your program must produce no output when run normally. From a402800f1f6409b861a5ef9434c9dddae83341fc Mon Sep 17 00:00:00 2001 From: SanjayTr <35617078+SanjayTr@users.noreply.github.com> Date: Fri, 26 Oct 2018 12:07:51 -0400 Subject: [PATCH 016/122] Updating instructions --- EducationalAssignments/ParityPartThree.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/EducationalAssignments/ParityPartThree.md b/EducationalAssignments/ParityPartThree.md index a7f71c50..a8803024 100644 --- a/EducationalAssignments/ParityPartThree.md +++ b/EducationalAssignments/ParityPartThree.md @@ -1,7 +1,7 @@ # Fixing your security layer -In this assignment you will analyze the bugs in your security layer from [[Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/ParityPartOne.md)] and fix them. You may want to use test cases from [Part Two](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/ParityPartTwo.md) to help identify these bugs. Finally, you will write a report discussing the different classes of bugs that your code had, and why. +In this assignment you will analyze the bugs in your security layer from [Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/ParityPartOne.md) and fix them. You may want to use test cases from [Part Two](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/ParityPartTwo.md) to help identify these bugs. Finally, you will write a report discussing the different classes of bugs that your code had, and why. @@ -55,5 +55,5 @@ Let's analyze one of the bugs in this implementation. According to the specific * Turn in the reference monitor that you have fixed. The name of the reference monitor should be in the form of reference_monitor_netId.r2py e.g. reference_monitor_jcappos.r2py All letters must be lowercase. - * In addition to your reference monitor, submit a one-page PDF document, in which you discuss the different classes of bugs you had in your reference monitors and why. + * In addition to your reference monitor, submit the one-time form whose link is shared in the assignments tab, in which you discuss the different classes of bugs you had in your reference monitors and why. What kind of bug did you experience, what caused the bug, and what did you do to fix it? From 72931dec0dea950135b453a3160787005ccd70e7 Mon Sep 17 00:00:00 2001 From: Adrian Date: Sun, 20 Jan 2019 06:48:00 -0500 Subject: [PATCH 017/122] Assignment clarification --- EducationalAssignments/ABStoragePartOne.md | 34 +++++++++++++++------- 1 file changed, 23 insertions(+), 11 deletions(-) diff --git a/EducationalAssignments/ABStoragePartOne.md b/EducationalAssignments/ABStoragePartOne.md index 1da473ec..3b92759c 100644 --- a/EducationalAssignments/ABStoragePartOne.md +++ b/EducationalAssignments/ABStoragePartOne.md @@ -28,19 +28,31 @@ recover if the file is written incorrectly. For this assignment, every 'E'. If any other characters (including lowercase 's', 'e', etc.) are the first or last characters, then the file is considered invalid. -However, you must permit the application to write information into the file. -The application should not be blocked from performing any writeat() operation, -because when it chooses it may later write 'S' at the start and 'E' at the -end. Note that checking if the file starts with 'S' and ends with 'E' is -only performed when close is called. +#### The Reference Monitor Must: +1. Allow all functionality of each method, per the list of [RepyV2 API calls](../Programming/RepyV2API.md) + *This includes creating new files + *Opening an existing file + *Reading valid file using readat() + *Writing to file using writeat(). The application should not be blocked +from performing any writeat() operation, because 'S' and 'E' may later be +written to the begining and end of the file respectively. + *Check if the file starts with 'S' and ends with 'E', only when close() is called. + +2. Not produce any errors + *Normal operations should not be blocked or produce any output + *Invalid operations should not produce any output to the user +#### The Reference Monitor Should: +1. Store two copies of the same file (filename.a and filename.b ) + *One is a valid backup, and the other that is written to +2. When an app calls ABopenfile(), this indicates that the A/B files, which + you should name filename.a and filename.b, should be opened. +3. When the app calls readat(), all reads must be performed on the valid file +4. Similarly, when the app calls writeat(), all writes must be performed +on the invalid file. + You may store two copies of A/B files on disk, one that is the valid backup -(which is used for reading) and the other that is written to. When an -app calls ABopenfile(), this indicates that the A/B files, which you should -name filename.a and filename.b, should be opened. -When the app calls readat(), all reads must be performed on the valid -file. Similarly, when the app calls writeat(), all writes must be -performed on the invalid file. If the app uses ABopenfile() to create a +file. If the app uses ABopenfile() to create a file that does not exist (by setting create=True when calling ABopenfile()), the reference monitor will create a new file 'SE' in filename.a and an empty file called filename.b. When close() is called on the file, if a file is From 3d4804df41de2a63ee2e33fb26aaa8ebeffd323b Mon Sep 17 00:00:00 2001 From: Adrian Date: Sun, 20 Jan 2019 08:36:07 -0500 Subject: [PATCH 018/122] Clarification of assignment --- EducationalAssignments/ABStoragePartOne.md | 100 ++++++++++----------- 1 file changed, 46 insertions(+), 54 deletions(-) diff --git a/EducationalAssignments/ABStoragePartOne.md b/EducationalAssignments/ABStoragePartOne.md index 3b92759c..f7a24fc5 100644 --- a/EducationalAssignments/ABStoragePartOne.md +++ b/EducationalAssignments/ABStoragePartOne.md @@ -16,52 +16,45 @@ reference monitors in a hands-on manner. - - ## Overview ---- In this assignment you will create a security layer which keeps a backup copy of a file in case it is written incorrectly. This is a common technique for things like firmware images where a system may not be able to -recover if the file is written incorrectly. For this assignment, every -`correct' file must start with the character 'S' and end with the character +recover if the file is written incorrectly. For this assignment, a +valid file must start with the character 'S' and end with the character 'E'. If any other characters (including lowercase 's', 'e', etc.) are the first or last characters, then the file is considered invalid. -#### The Reference Monitor Must: -1. Allow all functionality of each method, per the list of [RepyV2 API calls](../Programming/RepyV2API.md) - *This includes creating new files - *Opening an existing file - *Reading valid file using readat() - *Writing to file using writeat(). The application should not be blocked -from performing any writeat() operation, because 'S' and 'E' may later be -written to the begining and end of the file respectively. - *Check if the file starts with 'S' and ends with 'E', only when close() is called. - -2. Not produce any errors - *Normal operations should not be blocked or produce any output - *Invalid operations should not produce any output to the user -#### The Reference Monitor Should: -1. Store two copies of the same file (filename.a and filename.b ) - *One is a valid backup, and the other that is written to -2. When an app calls ABopenfile(), this indicates that the A/B files, which - you should name filename.a and filename.b, should be opened. -3. When the app calls readat(), all reads must be performed on the valid file -4. Similarly, when the app calls writeat(), all writes must be performed -on the invalid file. - - -You may store two copies of A/B files on disk, one that is the valid backup -file. If the app uses ABopenfile() to create a -file that does not exist (by setting create=True when calling ABopenfile()), -the reference monitor will create a new file 'SE' in filename.a and an empty +Applications use ABopenfile() to create or open a file. Files are created by +setting create=True when calling ABopenfile(), the reference +monitor will create a new file 'SE' in filename.a and an empty file called filename.b. When close() is called on the file, if a file is -not valid, it is discarded. if both files are valid, the older one is -discarded. +not valid, it is discarded. If both files are valid, the older one is +discarded. + +Write test applications to ensure your reference monitor behaves properly +in different cases and to test attacks against your monitor. + +#### The Reference Monitor Must: +1. Not modify or disable any functionality of any [RepyV2 API calls](../Programming/RepyV2API.md), such as: + * Creating new files + * Opening an existing file + * Reading valid file using readat() + * Writing to file using writeat(). This includes invalid writes, because 'S' and 'E' +may later be written to the begining and end of the file respectively. +2. Check if the file starts with 'S' and ends with 'E', only when close() is called. +3. Not produce any errors + * Normal operations should not be blocked or produce any output + * Invalid operations should not produce any output to the user +#### The Reference Monitor Should: +1. Store two copies of the same file (filename.a and filename.b) + * One is a valid backup, and the other is written to +2. When an app calls ABopenfile(), the method opens the A/B files, which + you should name filename.a and filename.b. +3. When the app calls readat(), all reads must be performed on the valid file +4. When the app calls writeat(), all writes must be performed on the invalid file. -Note that the behavior of other file system calls should remain unchanged. -This means listfiles(), removefile(), and calls to files accessed with -openfile() instead of ABopenfile() remain unchanged by this reference monitor. Three design paradigms are at work in this assignment: accuracy, efficiency, and security. @@ -88,12 +81,12 @@ Please refer to the [SeattleTestbed Build Instructions](../Contributing/BuildIns for details. Once you have built RepyV2 into a directory of your choice, change into that -directory. Use the command below in order to run your RepyV2 programs: +directory. Use the command below in order to run your RepyV2 applications: -```python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [program].r2py``` +```python2 repy.py restrictions.default encasementlib.r2py [security_layer].r2py [application].r2py``` -(Replace `[security_layer].r2py` and `[program].r2py` by the names of the -security layers and program that you want to run.) +(Replace '[security_layer].r2py' and '[application].r2py' by the names of the +security layers and application that you want to run.) In order to test whether or not these steps worked, please copy and paste the code found below for the sample security layer and sample attack. @@ -122,7 +115,7 @@ to run repy files. - -### Tutorials for Repy and Python + +### Tutorials for Repy and Python ---- Now that you have Repy and Python, you may need a refresher on how to use them. The following tutorials are excellent sources of information. * Python tutorial: **[http://docs.python.org/tutorial/]** * Seattle tutorial: **[https://seattle.poly.edu/wiki/PythonVsRepy]** * list of RepyV2 syntax: **[wiki:RepyV2API]** - -## Building the security layer + +## Building the security layer ---- **[wiki:RepyV2SecurityLayers]** explains the syntax used to build a reference monitor. The general tutorials above will aid in looking up other details about Repy. Remember, you have no idea how the attacker will try to penetrate your security layer, so it is important that you leave nothing to chance! Your reference monitor should try to stop every attack you can think of. Not just one or two. A good reference monitor will do this and incorporate the design paradigms of accuracy, efficiency and security defined above. - -### A basic (and inadequate) defense + +### A basic (and inadequate) defense Time to start coding! Let's inspect a basic security layer. @@ -218,19 +218,19 @@ secure_dispatch_module() ``` -### Using the example layer +### Using the example layer Keep in mind the above security layer would not prevent writing without permissions, and only protects reading without permissions from certain attacks. There are a bunch of tricks that can be used in order to circumvent this security layer easily. For instance, the above reference monitor isn't thread safe. For an introduction to thread safety please read [wiki/Thread_safety](http://en.wikipedia.org/wiki/Thread_safety). - -### Code analysis + +### Code analysis The above functions attempt to follow the design principles of accuracy, efficiency, and security. However they do so inadequately.For example the data will not be written at all unless both permissions to write and append are enabled. It is more important to make a security system that is infeasible to break, rather than impossible to break. However this security layer does not yet create the necessary infeasibility requirements for most attacks. Thus we see that there is a grey area of what is an acceptable level of impedance. - -### Testing your security layer + +### Testing your security layer ---- In this part of the assignment you will pretend to be an attacker. Remember the attacker's objective is to bypass permissions. By understanding how the attacker thinks, you will be able to write better security layers. Perhaps while attacking your security layer you will think of a new mitigation that should have been implemented. Keep in mind attacks are attempts to mitigate a given security protocol. If even one case succeeds, then your security layer has been compromised. Thus the attack you write should include several methods of attempting to bypass permissions. An example of an attack is found below: @@ -266,7 +266,7 @@ myfile.close() **Note:** All attacks should be written as Repy files, using the .r2py extension. -#### Code Analysis +#### Code Analysis It is important to keep in mind that only lowercase file names are allowed. So in the above code, specifically: ``` @@ -279,16 +279,16 @@ look.txt is a valid file name, however Look.txt is not. Examples of other invali This code attempts to read the “This is secure” from the file directly. First the file is opened using myfile=openfile("look.txt",True). Next myfile.writeat writes some data to the file with permissions. Then the write permissions are removed, and writeat is once again used to write some data to the file, some of which attempts to overwrite the existing data. If the security layer is written properly, only the appended data will be written. -### Running your security layer +### Running your security layer ---- Finally, type the following commands at the terminal to run your security layer with your attack program ```python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py ``` Make sure you went through the "How to get RepyV2" section! - -# Notes and Resources + +# Notes and Resources ---- * A list of command line utilities for windows can be found at **[http://commandwindows.com/command3.htm]** @@ -299,7 +299,7 @@ Make sure you went through the "How to get RepyV2" section! * For a complete list of syntax in Repyv2 please visit: **[wiki:RepyV2API]** - * The following link is an excellent source for information about security layers: **[http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf]** + * The following link is an excellent source for information about security layers: **[https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf]** * **[repy_v2/benchmarking-support/allnoopsec.py](https://seattle.poly.edu/browser/seattle/branches/repy_v2/benchmarking-support/allnoopsec.py)** is an empty security layer that doesn't perform any operations. @@ -312,15 +312,15 @@ Make sure you went through the "How to get RepyV2" section! **Your security layer should produce no output!! **If it encounters an attempt to read the secure data, you should raise a ValueError exception and allow execution to continue. * In repy log replaces print from python. This may be helpful when testing if Repy installed correctly. - -# Extra Credit + +# Extra Credit ---- Remember permissions for files after they are closed and reopened. This would include situations where your security layer is itself closed and reopened (for example, if the system restarts). You can assume that all reads must go through your security layer and so you can add an additional file or modify the file format. - -# What to turn in? + +# What to turn in? ---- * Turn in a repy file called reference_monitor_[ polyusername ].r2py with all letters in lowercase. Be sure your reference monitor never produces output. It should raise a ValueError if the calls should be blocked, but never, ever call log() to output information. Never raise unexpected errors. From bc203b342d86fe9344ecb3d35cca27be82cae196 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 17 Sep 2019 21:13:06 -0400 Subject: [PATCH 026/122] Update PrivateWritePartOne.md --- EducationalAssignments/PrivateWritePartOne.md | 58 +++++++++---------- 1 file changed, 29 insertions(+), 29 deletions(-) diff --git a/EducationalAssignments/PrivateWritePartOne.md b/EducationalAssignments/PrivateWritePartOne.md index 9ff931f0..e2ed95ed 100644 --- a/EducationalAssignments/PrivateWritePartOne.md +++ b/EducationalAssignments/PrivateWritePartOne.md @@ -1,5 +1,5 @@ -# Building a Security Layer +# Building a Security Layer This assignment will help you understand security mechanisms. You will be guided through the steps of creating a reference monitor using the security layer functionality in Repy V2. A reference monitor is an access control concept that refers to an abstract machine @@ -13,9 +13,9 @@ This assignment is intended to prepare you for thinking about security paradigms - -## Overview + +## Overview ---- In this assignment you will create a security layer which stops the attacker from reading data securely written in a file.You will create a function called **privatewrite** which allows the user to write data into a file which cannot be read back from the file.The data can be overwritten but isn't readable under any circumstance. You will do this by adding security rules to the functions available for reading from and writing to a file.Think of this as a method to write some data securely into a file. The future of the system depends on your ability to write secure code! @@ -26,15 +26,15 @@ Three design paradigms are at work in this assignment: accuracy, efficiency, and * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. This means you should not do things like re-read the file before each write. * Security: The attacker should not be able to circumvent the security layer. Hence, if the data written using privatewrite can be read back, the security is compromised. - -### Getting Python + +### Getting Python ---- Please note you must have Python 2.5, 2.6, or 2.7 to complete this assignment. Instructions on how to get Python for Windows can be found [InstallPythonOnWindows here]. If you are using Linux or a Mac it is likely you already have Python. In order to verify this, simply open a terminal and type ```python```. Please check its version on the initial prompt. **Note:**If you are using Windows, you will need python in your path variables. A tutorial on adding variables to your path in Windows can be found at [http://www.computerhope.com/issues/ch000549.htm] -### Getting RepyV2 +### Getting RepyV2 ---- The preferred way to get Repy is ''installing from source''. For this, you check out the required Git repositories, and run a build script. You can always update the repos later, and rebuild, so that you get the latest stable version of the Repy runtime. @@ -75,7 +75,7 @@ In order to test whether or not these steps worked, please copy and paste the co If you got an error, please go through the trouble shooting section found below. -### Trouble shooting Repy code +### Trouble shooting Repy code If you can't get Repy files to run, some of the following common errors may have occurred: * using `print` instead of `log`: @@ -99,24 +99,24 @@ To run the unit test, which will automatically tell you if you have errors with * [wiki:RepyV2CheckoutAndUnitTests] --> - -### Tutorials for Repy and Python + +### Tutorials for Repy and Python ---- Now that you have Repy and Python, you may need a refresher on how to use them. The following tutorials are excellent sources of information. * Python tutorial: **[http://docs.python.org/tutorial/]** * Seattle tutorial: **[https://seattle.poly.edu/wiki/PythonVsRepy]** * list of RepyV2 syntax: **[wiki:RepyV2API]** - -## Building the security layer + +## Building the security layer ---- **[wiki:RepyV2SecurityLayers]** explains the syntax used to build a reference monitor. The general tutorials above will aid in looking up other details about Repy. Remember, you have no idea how the attacker will try to penetrate your security layer, so it is important that you leave nothing to chance! Your reference monitor should try to stop every attack you can think of. Not just one or two. A good reference monitor will do this and incorporate the design paradigms of accuracy, efficiency and security defined above. - -### A basic (and inadequate) defense + +### A basic (and inadequate) defense Time to start coding! Let's inspect a basic security layer. @@ -183,22 +183,22 @@ CHILD_CONTEXT_DEF["openfile"] = {TYPE:OBJC,ARGS:(str,bool),EXCP:Exception,RETURN secure_dispatch_module() ``` -### Using the example layer +### Using the example layer Keep in mind the above security layer would only stop one kind of attack. Thus if an attacker doesn't know much about file input/output this will probably stop them. However there are a bunch of tricks that can be used in order to circumvent this security layer easily. For instance, the above reference monitor isn't thread safe. For an introduction to thread safety please read [wiki/Thread_safety](http://en.wikipedia.org/wiki/Thread_safety). - -### Code analysis + +### Code analysis The `privatewrite()` function attempts to follow the design principles of accuracy, efficiency, and security. However it does so inadequately.For example the data is blocked even if its overwritten using a normal `writeat()` function. It is more important to make a security system that is infeasible to break, rather than impossible to break. However this security layer does not yet create the necessary infeasibility requirements for most attacks. Thus we see that there is a grey area of what is an acceptable level of impedance. Looking at the structure of `privatewrite()`,the flag `privatedata` is set when data is written using `privatewrite()`. During the `readat()` function, the flag is checked against and accordingly the data is either displayed or an exception is raised. - -### Testing your security layer + +### Testing your security layer ---- In this part of the assignment you will pretend to be an attacker. Remember the attacker's objective is to read the `secure data` written in the file. By understanding how the attacker thinks, you will be able to write better security layers. Perhaps while attacking your security layer you will think of a new mitigation that should have been implemented. Keep in mind attacks are attempts to mitigate a given security protocol. If even one case succeeds, then your security layer has been compromised. Thus the attack you write should include several methods of attempting to read the secure data from the file. An example of an attack is found below: ``` @@ -233,7 +233,7 @@ finally: **Note:** All attacks should be written as Repy files, using the `.r2py` extension. -#### Code Analysis +#### Code Analysis It is important to keep in mind that only lowercase file names are allowed. So in the above code, specifically: ``` @@ -245,18 +245,18 @@ myfile=openfile("look.txt",True) look.txt is a valid file name, however Look.txt is not. Examples of other invalid files names are, look@.txt, look/.txt, and look().txt. Essentially all non-alphanumeric characters are not allowed. This code attempts to read the `This is secure` from the file directly. First the file is opened using -`myfile=openfile("look.txt",True)`. Next `myfile.writeat` writes some data to the file. Then the `privatewrite` function is used to write some data securely to the file. The 0 refers to an offset of zero. The `try:` statement tells the program to "try" this case. Notice that the `except` is executed if an error is raised. If the security layer fails the test then the else statement is executed. The `finally:` statement will always run, closing the file. +`myfile=openfile("look.txt",True)`. Next `myfile.writeat` writes some data to the file. Then the `privatewrite` function is used to write some data securely to the file. The 0 refers to an offset of zero. The `try:` statement tells the program to "try" this case. Notice that the `except` is executed if an error is raised. If the security layer fails the test then the else statement is executed. The `finally:` statement will always run, closing the file. -### Running your security layer +### Running your security layer ---- Finally, type the following commands at the terminal to run your security layer with your attack program ```python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py ``` Make sure you went through the "How to get RepyV2" section! - -# Notes and Resources + +# Notes and Resources ---- * A list of command line utilities for windows can be found at **[http://commandwindows.com/command3.htm]** @@ -267,7 +267,7 @@ Make sure you went through the "How to get RepyV2" section! * For a complete list of syntax in Repyv2 please visit: **[wiki:RepyV2API]** - * The following link is an excellent source for information about security layers: **[http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf]** + * The following link is an excellent source for information about security layers: **[https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf]** * **[repy_v2/benchmarking-support/allnoopsec.py](https://seattle.poly.edu/browser/seattle/branches/repy_v2/benchmarking-support/allnoopsec.py)** is an empty security layer that doesn't perform any operations. @@ -280,17 +280,17 @@ Make sure you went through the "How to get RepyV2" section! **Your security layer should produce no output!! **If it encounters an attempt to read the secure data, you should raise a `ValueError` exception and allow execution to continue. * In repy log replaces print from python. This may be helpful when testing if Repy installed correctly. - -# Extra Credit + +# Extra Credit ---- For extra credit - Protect the file against read and write operations involving persistent data. If you close the file and open it again the system should be able to block the future attacks. - -# What to turn in? + +# What to turn in? ---- * Turn in a repy file called reference_monitor_[ name or id number].r2py. Be sure your reference monitor **never produces output**. It should raise a ValueError if the readat should be blocked, but never, ever call log() to output information or raise unexpected errors. - * For extra credit turn in a second repy file called extra_credit_[ name or id number].r2py \ No newline at end of file + * For extra credit turn in a second repy file called extra_credit_[ name or id number].r2py From 45fc7fd064e6e0e640577a72864ad9b512fcb868 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 17 Sep 2019 21:13:39 -0400 Subject: [PATCH 027/122] Update PrivateWritePartTwo.md --- EducationalAssignments/PrivateWritePartTwo.md | 56 +++++++++---------- 1 file changed, 28 insertions(+), 28 deletions(-) diff --git a/EducationalAssignments/PrivateWritePartTwo.md b/EducationalAssignments/PrivateWritePartTwo.md index 3603c9f6..3ef7eb4d 100644 --- a/EducationalAssignments/PrivateWritePartTwo.md +++ b/EducationalAssignments/PrivateWritePartTwo.md @@ -1,12 +1,12 @@ -# Security layer testing and penetration +# Security layer testing and penetration In this assignment you will learn how to attack a reference monitor. The reference monitor you will be testing uses the security layer framework (encasement library, etc.) for the Seattle testbed. It is possible to do this assignment separately however it is recommended that this assignment be completed after [wiki:EducationalAssignments/PrivateWritePartOne PrivateWritePartOne]. Either way you should already have a working security layer or access to one. Testing the security layer is done by running a series of test cases that an adversary may use to circumvent your system. This assignment is intended to prepare you for thinking about security paradigms in a functional way. The ideas of information, security and privacy have been embedded into the steps of this assignment. - -## Overview + +## Overview ---- In this assignment you are a tester. You have been sent a bunch of reference monitors that need testing before they are deployed. Your job will be to ensure an attacker cannot circumvent these security layers. In order to do this, you will attempt to read the secure data written in the file using the function `privatewrite()`. If you are able to do so, then the security layer is not secure. The future of the system depends on your ability to test code thoroughly! @@ -28,24 +28,24 @@ Within the context of this assignment these design paradigms translate to: You will submit a zip file containing all of the tests you have created. You will gain points for every student's reference monitor you find a flaw in. It is okay if multiple tests of yours breaking a student's reference monitor, but you will not gain additional points. - -## Prerequisites + +## Prerequisites This assignment assumes you have Python2.5 or Python2.6, Repy and RepyV2 installed on your computer. If you don't already have them please go to [wiki:EducationalAssignments/PrivateWritePartOne#GettingPythonRepy PrivateWritePartOne] for a tutorial on how to get them. - -### Helpful links + +### Helpful links ---- The following links will aid students in becoming comfortable with Python, Repy and seattle: * Python tutorial: **[http://docs.python.org/tutorial/]** * Seattle tutorial: **[https://seattle.poly.edu/wiki/PythonVsRepy]** * list of RepyV2 syntax: **[wiki:RepyV2API]** - -## Testing security layers + +## Testing security layers ---- -### Hypothesis, test case, counter example +### Hypothesis, test case, counter example The goal of a good tester is to test hypotheses. A hypothesis is just a scientific way of asking a question. The hypothesis of this assignment is "This security layer is well designed." The questions you will ask when running your test cases will always be the same @@ -59,11 +59,11 @@ Notice that these questions are parallels of the security paradigms: security, e If we can find a case where the hypothesis is false, then the security layer is not secure. Such a case is referred to as a counter example. Hence all test cases should be designed to test for these three types of flaws. - -### Examples of tests + +### Examples of tests Test cases are briefly described in [wiki:EducationalAssignments/PrivateWritePartOne PrivateWritePartOne] and [wiki:RepyV2SecurityLayers]. Below is another example of a test case you may want to consider. This test case gives the right 'style' for your all your test cases, but lacks in the number of test cases. A good attack will include many test cases. -#### Test case 1: +#### Test case 1: ``` # Open a file @@ -88,7 +88,7 @@ finally: # Close the file after our attempt. myfile.close() ``` -#### Code analysis +#### Code analysis It is important to keep in mind that only lowercase file names are allowed. So in the above code, specifically: ``` @@ -101,7 +101,7 @@ look.txt is a valid file name, however Look.txt is not. Examples of other inval In this case we are verifying the security of the reference monitor. This code attempts to read the data written on the first offset value. First the file is opened using `myfile=openfile("look.txt",True)`. Next `myfile.readat(1,0)` tries to read from the file. The 0 refers to an offset of zero and 1 refers to number of characters being read. The try: statement tells the program to "try" this case. Notice that the except is executed if an error is raised. If the security layer fails the test then the else statement is executed. The finally: statement will always run, closing the file. -#### Test case 2: +#### Test case 2: ``` # Open a file @@ -124,20 +124,20 @@ finally: # Close the file after our attempt. myfile.close() ``` -#### Code analysis +#### Code analysis In this case we are verifying the accuracy of the reference monitor. This code attempts to write `"abcd"` to the file directly. We assume that secure data is written in the file on the first four offset values. First the file is opened using `myfile=openfile("look.txt",True)`. Next `myfile.writeat("abcd",0)` tries to write `"abcd"` to the file. The 0 refers to an offset of zero. The try: statement tells the program to "try" this case. Notice that the except is executed if an error is raised. If the security layer fails the test then the else statement is executed. The finally: statement will always run, closing the file. If this case produces anything other than "Write okay", then this layer fails the accuracy design paradigm. The security layer should only stop a file from reading the secure data not overwriting it. -#### More information on: Try, Except, Else, Finally +#### More information on: Try, Except, Else, Finally The try, except, else and finally statements are part of **exception handling**. For more information on exception handling please visit: * [http://docs.python.org/tutorial/errors.html] * [http://wiki.python.org/moin/HandlingExceptions] * [http://www.tutorialspoint.com/python/python_exceptions.htm] -### Hints and Ideas for testing +### Hints and Ideas for testing When writing your own tests it is important to test for a complete set of possible penetrations. Keep in mind, it only takes one test case to break through a security layer. Some of the things you may want to test for include: @@ -146,12 +146,12 @@ When writing your own tests it is important to test for a complete set of possib * writing to both positions at once And more! Remember a good security layer can't be broken by anyone! Which is all part of the fun! It's about solving a puzzle. First you make the puzzle - write the security layer, then you solve the puzzle - try to bypass it. If your puzzle is "good enough", no one will be able to break it, no matter what. - -## Notes and Resources + +## Notes and Resources ---- - * The following link is an excellent source for information about security layers: http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf + * The following link is an excellent source for information about security layers: https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf * [repy_v2/benchmarking-support/allnoopsec.py](https://seattle.poly.edu/browser/seattle/branches/repy_v2/benchmarking-support/allnoopsec.py) is an empty security layer that doesn't perform any operations. @@ -160,15 +160,15 @@ And more! Remember a good security layer can't be broken by anyone! Which is a * In repy 'log' replaces 'print' from python. Many students find this to be a stumbling block. - -## Extra Credit + +## Extra Credit ---- Find bugs in the extra credit reference monitors given the altered threat model. You should include more test cases in the extra credit! - -## How to run your tests on many reference monitors + +## How to run your tests on many reference monitors ---- If you are using Mac or Linux, you can do something like the following: @@ -185,11 +185,11 @@ This will print out the output from each program. If you are a Windows user and you create your own solution that does the same thing, please post it on the forum. - -## What to turn in? + +## What to turn in? ---- * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must start with your first and last name in lowercase separated by underscores: first_last_. For example: justin_cappos_securitytest1.repy justincappos_goodaccuracytest.repy are both valid names. * Optionally turn in the test cases used to attack the extra credit reference monitors in a zip file. Note that in this case, you can expect that your code is run more than once. In the name of the file, say if it needs to be run multiple times. For example: justin_cappos_run_twice_metadata_removal.repy justincappos_run_once_threading_hack.repy. - \ No newline at end of file + From d9082a747dbbffcc12d2345f9e21c0269243ec43 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 17 Sep 2019 21:14:07 -0400 Subject: [PATCH 028/122] Update SetMaxFileSizePartOne.md --- .../SetMaxFileSizePartOne.md | 52 +++++++++---------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/EducationalAssignments/SetMaxFileSizePartOne.md b/EducationalAssignments/SetMaxFileSizePartOne.md index 9a51c08d..65a204d3 100644 --- a/EducationalAssignments/SetMaxFileSizePartOne.md +++ b/EducationalAssignments/SetMaxFileSizePartOne.md @@ -1,4 +1,4 @@ -# Implement a Defensive Security System +# Implement a Defensive Security System This assignment will help you understand security mechanisms. You will be guided through the steps of creating a reference monitor using the security layer functionality in Repy V2. A reference monitor is an access control concept that refers to an abstract machine that mediates all access to objects by subjects. This can be used to allow, deny, or change the behavior of any set of calls. While not a perfect way of validating your reference monitor, it is useful to create test cases to see whether your security layer will work as expected. (The test cases may be turned in as part of the next assignment.) @@ -7,9 +7,9 @@ This assignment is intended to reinforce concepts about access control and refer - -## Overview + +## Overview ---- In this assignment you will create a security layer which stops the user from writing to a file after a specified file size. You will add a function called setmaxfilesize() that will take a file size as parameter and enforce it for the specified file. If a file is longer than the specified size, any data past this point in the file will be truncated. (Since the Repy V2 API does not have a truncate call, you will have to implement this functionality yourself.) If a writeat() call is performed at a location at or past the end of the file size limit, a SeekPastEndOfFileError should be raised. If the writeat() starts before the file limit, any data up to the size limit should be written and all other data should be discarded. @@ -22,15 +22,15 @@ Three design paradigms are at work in this assignment: accuracy, efficiency, and * Security: The attacker should not be able to circumvent the security layer. Hence, if the data can be written beyond specified size, the security is compromised. - -### Getting Python + +### Getting Python ---- Please note you must have Python 2.5, 2.6, or 2.7 to complete this assignment. Instructions on how to get Python for Windows can be found [InstallPythonOnWindows here]. If you are using Linux or a Mac it is likely you already have Python. In order to verify this, simply open a terminal and type ```python```. Please check its version on the initial prompt. **Note:**If you are using Windows, you will need python in your path variables. A tutorial on adding variables to your path in Windows can be found at [http://www.computerhope.com/issues/ch000549.htm] -### Getting RepyV2 +### Getting RepyV2 ---- The preferred way to get Repy is ''installing from source''. For this, you check out the required Git repositories, and run a build script. You can always update the repos later, and rebuild, so that you get the latest stable version of the Repy runtime. @@ -71,7 +71,7 @@ In order to test whether or not these steps worked, please copy and paste the co If you got an error, please go through the trouble shooting section found below. -### Troubleshooting Repy code +### Troubleshooting Repy code ---- If you can't get Repy files to run, some of the following common errors may have occurred: @@ -96,24 +96,24 @@ To run the unit test, which will automatically tell you if you have errors with * [wiki:RepyV2CheckoutAndUnitTests] --> - -### Tutorials for Repy and Python + +### Tutorials for Repy and Python ---- Now that you have Repy and Python, you may need a refresher on how to use them. The following tutorials are excellent sources of information. * Python tutorial: **[http://docs.python.org/tutorial/]** * Seattle tutorial: **[https://seattle.poly.edu/wiki/PythonVsRepy]** * list of RepyV2 API calls: **[wiki:RepyV2API]** - -## Building the security layer + +## Building the security layer ---- The following program is a basic and incomplete sample code for you to get an idea about writing security layer. Remember, you have no idea how the attacker will try to penetrate your security layer, so it is important that you leave nothing to chance! - -### A basic (and inadequate) defense + +### A basic (and inadequate) defense Time to start coding! Let's inspect a basic security layer. @@ -196,9 +196,9 @@ CHILD_CONTEXT_DEF["removefile"]["target"] = secure_removefile secure_dispatch_module() ``` - -### Testing your security layer + +### Testing your security layer ---- In this part of the assignment you will pretend to be an attacker. Remember the attacker's objective is to bypass the file size restrictions or cause the security layer to act in a disallowed manner. By understanding how the attacker thinks, you will be able to write better security layers. @@ -220,7 +220,7 @@ try: except SeekPastEndOfFileError: pass # there should be an exception here... else: - log("ERROR! Didn't stop me from writing past the maximum file size. + log("ERROR! Didn't stop me from writing past the maximum file size. n") @@ -230,7 +230,7 @@ myfile.close() **Note:** All attacks should be written as Repy V2 files, using the .r2py extension. -#### Choice of File Names +#### Choice of File Names ---- It is important to keep in mind that only lowercase file names are allowed. So in the above code, specifically: @@ -241,21 +241,21 @@ myfile=openfile("look.txt",True) look.txt is a valid file name, however Look.txt is not. Examples of other invalid files names are, look@.txt, look/.txt, and look().txt. Essentially all non-alphanumeric characters are not allowed. -### Running your security layer +### Running your security layer ---- Finally, type the following commands at the terminal to run your security layer with your attack program ```python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py ``` Make sure you went through the "How to get RepyV2" section! - -# Notes and Resources + +# Notes and Resources ---- * For a complete list of syntax in Repyv2 please visit: **[wiki:RepyV2API]** - * The following link is an excellent source for information about security layers: **[http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf]** + * The following link is an excellent source for information about security layers: **[https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf]** * **Note:** It is possible to add multiple security layers to Repy, this may be useful for testing different mitigations separately. This is done with the following command at the terminal: @@ -264,17 +264,17 @@ Make sure you went through the "How to get RepyV2" section! **Your security layer should produce no output!! ** * In repy log replaces print from python. This may be helpful when testing if Repy installed correctly. - -# Extra Credit + +# Extra Credit ---- For extra credit, make a second security layer which retains its specified size limits even if it is closed and reopened again. The file size limit should stay after the program is terminated and restarted Do not submit this code inside your assignment. Submit a separate copy for extra credit. - -# What to turn in? + +# What to turn in? ---- * Turn in a repy file called reference_monitor_[ netid ].r2py with all letters in lowercase. * **Never raise unexpected errors or produce any output.** Your program must produce no output when run normally. - * For extra credit turn in a second repy file called extra_credit_[netid].r2py **You must turn in separate files for the normal assignment and extra credit** \ No newline at end of file + * For extra credit turn in a second repy file called extra_credit_[netid].r2py **You must turn in separate files for the normal assignment and extra credit** From 99696b63f772342b64f190605e1a7951f69edd46 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 17 Sep 2019 21:15:20 -0400 Subject: [PATCH 029/122] Update ParityPartTwo.md --- EducationalAssignments/ParityPartTwo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/ParityPartTwo.md b/EducationalAssignments/ParityPartTwo.md index a3971b00..67349e0c 100644 --- a/EducationalAssignments/ParityPartTwo.md +++ b/EducationalAssignments/ParityPartTwo.md @@ -158,7 +158,7 @@ And more! Remember a good security layer can't be broken by anyone! Which is a ## Notes and Resources ---- - * The following link is an excellent source for information about security layers: http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf + * The following link is an excellent source for information about security layers: https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf * [repy_v2/benchmarking-support/allnoopsec.py](https://seattle.poly.edu/browser/seattle/branches/repy_v2/benchmarking-support/allnoopsec.py) is an empty security layer that doesn't perform any operations. From 384762a320341da5fe6f76e025601e9bf2bd8148 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 17 Sep 2019 21:15:44 -0400 Subject: [PATCH 030/122] Update ParityPartOne.md --- EducationalAssignments/ParityPartOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/ParityPartOne.md b/EducationalAssignments/ParityPartOne.md index 99d732ab..5c8b5c68 100644 --- a/EducationalAssignments/ParityPartOne.md +++ b/EducationalAssignments/ParityPartOne.md @@ -305,7 +305,7 @@ Make sure you went through the "How to get RepyV2" section! * For a complete list of syntax in Repyv2 please visit the **[RepyV2 API documentation](https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md)** - * **[This paper](http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf)** + * **[This paper](https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf)** is an excellent source for information about security layers * **Note:** It is possible to add multiple security layers to Repy, this From 0f490c9eb7140951d3373b625d27bc9889838f2d Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 17 Sep 2019 21:16:22 -0400 Subject: [PATCH 031/122] Update SecurityLayerPartTwo.md --- .../SecurityLayerPartTwo.md | 52 +++++++++---------- 1 file changed, 26 insertions(+), 26 deletions(-) diff --git a/EducationalAssignments/SecurityLayerPartTwo.md b/EducationalAssignments/SecurityLayerPartTwo.md index 3e8c730f..92ec63e1 100644 --- a/EducationalAssignments/SecurityLayerPartTwo.md +++ b/EducationalAssignments/SecurityLayerPartTwo.md @@ -1,12 +1,12 @@ -# Security layer testing and penetration +# Security layer testing and penetration In this assignment you will learn how to attack a reference monitor. The reference monitor you will be testing uses the security layer framework (encasement library, etc.) for the Seattle testbed. It is possible to do this assignment separately however it is recommended that this assignment be you completed after [wiki:EducationalAssignments/SecurityLayerPartOne SecurityLayerPartOne]. Either way you should already have a working security layer or access to one. Testing the security layer is done by running a series of test cases that an adversary may use to circumvent your system. This assignment is intended to prepare you for thinking about security paradigms in a functional way. The ideas of information, security and privacy have been embedded into the steps of this assignment. - -## Overview + +## Overview ---- In this assignment you are a tester. You have been sent a bunch of reference monitors that need testing before they are deployed. Your job will be to ensure an attacker cannot circumvent these security layers. In order to do this, you will attempt to write "MZ" to the first two characters of a file. If you are able to do so, then the security layer is not secure. The future of the system depends on your ability to test code thoroughly! @@ -31,24 +31,24 @@ After testing, write a report on your findings, including all attacks that succe Note: For a full description on MZ executable files see http://www.fileformat.info/format/exe/corion-mz.htm - -## Prerequisites + +## Prerequisites This assignment assumes you have Python2.5 or Python2.6, Repy and RepyV2 installed on your computer. If you don't already have them please go to [wiki:EducationalAssignments/SecurityLayerPartOne#GettingPythonRepy SecurityLayerPartOne] for a tutorial on how to get them. - -### Helpful links + +### Helpful links ---- The following links will aid students in becoming comfortable with Python, Repy and seattle: * Python tutorial: **[http://docs.python.org/tutorial/]** * Seattle tutorial: **[https://seattle.poly.edu/wiki/PythonVsRepy]** * list of RepyV2 syntax: **[wiki:RepyV2API]** - -## Testing security layers + +## Testing security layers ---- -### Hypothesis, test case, counter example +### Hypothesis, test case, counter example The goal of a good tester is to test hypotheses. A hypothesis is just a scientific way of asking a question. The hypothesis of this assignment is "This security layer is well designed." The questions you will ask when running your test cases will always be the same @@ -62,11 +62,11 @@ Notice that these questions are parallels of the security paradigms: security, e If we can find a case where the hypothesis is false, then the security layer is not secure. Such a case is referred to as a counter example. Hence all test cases should be designed to test for these three types of flaws. - -### Examples of tests + +### Examples of tests Test cases are briefly described in [wiki:EducationalAssignments/SecurityLayerPartOne SecurityLayerPartOne] and [wiki:RepyV2SecurityLayers]. Below is another example of a test case you may want to consider. This test case gives the right 'style' for your all your test cases, but lacks in the number of test cases. A good attack will include many test cases. -#### Test case 1: +#### Test case 1: ``` # Open a file @@ -88,7 +88,7 @@ finally: # Close the file after our attempt. myfile.close() ``` -#### Code analysis +#### Code analysis It is important to keep in mind that only lowercase file names are allowed. So in the above code, specifically: ``` @@ -101,7 +101,7 @@ look.txt is a valid file name, however Look.txt is not. Examples of other inval In this case we are verifying the security of the reference monitor. This code attempts to write "MZ" to the file directly. First the file is opened using myfile=openfile("look.txt",True). Next myfile.writeat("MZ",0) tries to write "MZ" to the file. The 0 refers to an offset of zero. The try: statement tells the program to "try" this case. Notice that the except is executed if an error is raised. If the security layer fails the test then the else statement is executed. The finally: statement will always run, closing the file. -#### Test case 2: +#### Test case 2: ``` # Open a file @@ -123,20 +123,20 @@ finally: # Close the file after our attempt. myfile.close() ``` -#### Code analysis +#### Code analysis In this case we are verifying the accuracy of the reference monitor. This code attempts to write "MZ" to the file directly. First the file is opened using myfile=openfile("look.txt",True). Next myfile.writeat("MZ",1) tries to write "MZ" to the file. The 1 refers to the fact that the characters are placed as the second and third characters in the file. The try: statement tells the program to "try" this case. Notice that the except is executed if an error is raised. If the security layer fails the test then the else statement is executed. The finally: statement will always run, closing the file. If this case produces anything other than "Write okay", then this layer fails the accuracy design paradigm. The security layer should only stop a file from starting with "MZ", not containing "MZ". -#### More information on: Try, Except, Else, Finally +#### More information on: Try, Except, Else, Finally The try, except, else and finally statements are part of **exception handling**. For more information on exception handling please visit: * [http://docs.python.org/tutorial/errors.html] * [http://wiki.python.org/moin/HandlingExceptions] * [http://www.tutorialspoint.com/python/python_exceptions.htm] -### Hints and Ideas for testing +### Hints and Ideas for testing When writing your own tests it is important to test for a complete set of possible penetrations. Keep in mind, it only takes one test case to break through a security layer. Some of the things you may want to test for include: @@ -145,12 +145,12 @@ When writing your own tests it is important to test for a complete set of possib * writing to both positions at once And more! Remember a good security layer can't be broken by anyone! Which is all part of the fun! It's about solving a puzzle. First you make the puzzle - write the security layer, then you solve the puzzle - try to bypass it. If your puzzle is "good enough", no one will be able to break it, no matter what. - -## Notes and Resources + +## Notes and Resources ---- - * The following link is an excellent source for information about security layers: http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf + * The following link is an excellent source for information about security layers: https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf * [repy_v2/benchmarking-support/allnoopsec.py](https://seattle.poly.edu/browser/seattle/branches/repy_v2/benchmarking-support/allnoopsec.py) is an empty security layer that doesn't perform any operations. @@ -159,16 +159,16 @@ And more! Remember a good security layer can't be broken by anyone! Which is a * In repy 'log' replaces 'print' from python. Many students find this to be a stumbling block. - -## Extra Credit + +## Extra Credit ---- Write and test a second reference monitor that stops "p0wnd" from being written anywhere in a file. You should include more test cases in the extra credit! - -## What to turn in? + +## What to turn in? ---- * Turn in the test cases used to attack a given reference monitor. * Turn in the write up of each reference monitor attacked, with critique of the reference monitor. Critique should include all test cases that worked. - * Write up for 'p0wnd' reference monitors. \ No newline at end of file + * Write up for 'p0wnd' reference monitors. From a3cb722dbfa76796fe8ccc977f905628b4439859 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 17 Sep 2019 21:16:52 -0400 Subject: [PATCH 032/122] Update ProtectFilePartOne.md --- EducationalAssignments/ProtectFilePartOne.md | 64 ++++++++++---------- 1 file changed, 32 insertions(+), 32 deletions(-) diff --git a/EducationalAssignments/ProtectFilePartOne.md b/EducationalAssignments/ProtectFilePartOne.md index 28dd6c91..eeff5de5 100644 --- a/EducationalAssignments/ProtectFilePartOne.md +++ b/EducationalAssignments/ProtectFilePartOne.md @@ -1,4 +1,4 @@ -# Implement a Defensive Security System +# Implement a Defensive Security System This assignment will help you understand security mechanisms. You will be guided through the steps of creating a reference monitor using the security layer functionality in Repy V2. A reference monitor is an access control concept that refers to an abstract machine that mediates all access to objects by subjects. This can be used to allow, deny, or change the behavior of any set of calls. While not a perfect way of validating your reference monitor, it is useful to create test cases to see whether your security layer will work as expected. (The test cases may be turned in as part of the next assignment.) @@ -7,9 +7,9 @@ This assignment is intended to reinforce concepts about access control and refer - -## Overview + +## Overview ---- In this assignment you will create a reference monitor that will, when asked to do so, protect files starting with private_ (and only those files). If the contents of a file begin with the characters “INV”, then a listfiles() call **must not** show that the file exists. If the contents of a file begin with the characters “PER” then a removefile() call **must** raise a RepyArgumentError. In cases not specified above, all calls must work identically to the RepyV2 API. @@ -20,18 +20,18 @@ Three design paradigms are at work in this assignment: accuracy, efficiency, and * Security: The attacker should not be able to circumvent the reference monitor and cause files that should not be displayed to be displayed or files that should not be removed to be removed. - -### Getting Python + +### Getting Python ---- Please note you must have Python 2.5, 2.6, or 2.7 to complete this assignment. Instructions on how to get Python for Windows can be found [InstallPythonOnWindows here]. If you are using Linux or a Mac it is likely you already have Python. In order to verify this, simply open a terminal and type ```python```. Please check its version on the initial prompt. **Note:**If you are using Windows, you will need python in your path variables. A tutorial on adding variables to your path in Windows can be found at [http://www.computerhope.com/issues/ch000549.htm] - -### Getting RepyV2 + +### Getting RepyV2 ---- The preferred way to get Repy is ''installing from source''. For this, you check out the required Git repositories, and run a build script. You can always update the repos later, and rebuild, so that you get the latest stable version of the Repy runtime. @@ -73,9 +73,9 @@ In order to test whether or not these steps worked, please copy and paste the co If you got an error, please go through the troubleshooting section found below. - -### Troubleshooting Repy code + +### Troubleshooting Repy code ---- If you can't get Repy files to run, some of the following common errors may have occurred: @@ -102,9 +102,9 @@ To run the unit test, which will automatically tell you if you have errors with --> - -### Tutorials for Repy and Python + +### Tutorials for Repy and Python ---- Now that you have Repy and Python, you may need a refresher on how to use them. The following tutorials are excellent sources of information. @@ -113,16 +113,16 @@ Now that you have Repy and Python, you may need a refresher on how to use them. * List of RepyV2 API calls: **[wiki:RepyV2API]** - -## Building the security layer + +## Building the security layer ---- The following program is a basic and incomplete sample code for you to get an idea about writing security layer. Remember, you have no idea how the attacker will try to penetrate your security layer, so it is important that you leave nothing to chance! - -### A basic (and inadequate) defense + +### A basic (and inadequate) defense ---- Time to start coding! Let's inspect a basic security layer. @@ -168,7 +168,7 @@ def secure_openfile(filename, create): def secure_removefile(filename): if filename in PERBUFFER: - raise RepyArgumentError("Cannot delete this file! + raise RepyArgumentError("Cannot delete this file! n") removefile(filename) @@ -219,9 +219,9 @@ secure_dispatch_module() ``` - -### Testing your security layer + +### Testing your security layer ---- In this part of the assignment you will pretend to be an attacker. Remember the attacker's objective is to bypass the restrictions or cause the security layer to act in a disallowed manner. By understanding how the attacker thinks, you will be able to write better security layers. @@ -238,7 +238,7 @@ myfile = openfile("private_testfile.txt", True) myfile.writeat("INV", 0) if "private_testfile.txt" in listfiles(): - log("ERROR! File not invisible. + log("ERROR! File not invisible. n") # Write into the file again @@ -252,16 +252,16 @@ try: except RepyArgumentError: pass # There should be an exception here! else: - log("ERROR! Allowed me to delete the file. + log("ERROR! Allowed me to delete the file. n") ``` **Note:** All attacks should be written as Repy V2 files, using the .r2py extension. - -#### Choice of File Names + +#### Choice of File Names ---- It is important to keep in mind that only lowercase file names are allowed. So in the above code, specifically: @@ -273,9 +273,9 @@ myfile=openfile("look.txt",True) look.txt is a valid file name, however Look.txt is not. Examples of other invalid files names are, look@.txt, look/.txt, and look().txt. Essentially all non-alphanumeric characters are not allowed. - -### Running your security layer + +### Running your security layer ---- Finally, type the following commands at the terminal to run your security layer with your attack program @@ -284,14 +284,14 @@ Finally, type the following commands at the terminal to run your security layer Make sure you went through the "How to get RepyV2" section! - -## Notes and Resources + +## Notes and Resources ---- * For a complete list of syntax in Repyv2 please visit: **[wiki:RepyV2API]** - * The following link is an excellent source for information about security layers: **[http://isis.poly.edu/~jcappos/papers/cappos_seattle_ccs_10.pdf]** + * The following link is an excellent source for information about security layers: **[https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf]** * It is possible to add multiple security layers to Repy, this may be useful for testing different mitigations separately. This is done with the following command at the terminal: @@ -302,21 +302,21 @@ Make sure you went through the "How to get RepyV2" section! * In repy log replaces print from python. This may be helpful when testing if Repy installed correctly. - -## Extra Credit + +## Extra Credit ---- For extra credit, make a second security layer which will prevent writing to a private_ file when the first and last three characters in the file **match** and are either "INV" or "PER". If the first three characters in a private_ file are "INV" and the last three are "PER", you should be able to write to the file. Note: Do not submit this code inside your assignment file. Submit a separate copy for extra credit. - -## What to turn in? + +## What to turn in? ---- * Turn in a repy file called reference_monitor_[netid].r2py with all letters in lowercase. * **Never raise unexpected errors or produce any output.** Your program must produce no output when run normally. - * For extra credit turn in a second repy file called extra_credit_[netid].r2py **You must turn in separate files for the normal assignment and extra credit** \ No newline at end of file + * For extra credit turn in a second repy file called extra_credit_[netid].r2py **You must turn in separate files for the normal assignment and extra credit** From f43d8e12dfc43b124917c08d9abcd3ed93f826a0 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 22 Sep 2019 04:26:53 -0400 Subject: [PATCH 033/122] Overview edits Made assignment more clear. I decided to make it so that keeping the backup files are not required because I believe keeping the readonly file can create a vulnerability where if someone wrote a program to read every file in the directory it would eventually try to read a .a file and generate a .a.a and so forth which will effectively create an infinite loop as well as fill up memory. I can change that to just be discards filename.b if that is the intent. I think keeping the .b should not cause any problems. I think this part can be kept ambiguous to see if anyone tries to pull off any attacks based on keeping the files and helps with the unifying theme of the "backup file". It also ties in with line 51 where a valid backup is mentioned. --------- In line 54, valid file and invalid file is a bit confusing. Esp when if a file is written to correctly it wouldn't be invalid. Also might lead students to do inefficient checks on both files to see which file is valid and which is invalid before read write. --- EducationalAssignments/ABStoragePartOne.md | 27 +++++++++++----------- 1 file changed, 13 insertions(+), 14 deletions(-) diff --git a/EducationalAssignments/ABStoragePartOne.md b/EducationalAssignments/ABStoragePartOne.md index 8fd11dab..b7e14555 100644 --- a/EducationalAssignments/ABStoragePartOne.md +++ b/EducationalAssignments/ABStoragePartOne.md @@ -28,10 +28,8 @@ the first or last characters, then the file is considered invalid. Applications use ABopenfile() to create or open a file. Files are created by setting create=True when calling ABopenfile(), the reference -monitor will create a new file 'SE' in filename.a and an empty -file called filename.b. When close() is called on the file, if a file is -not valid, it is discarded. If both files are valid, the older one is -discarded. +monitor will create a valid backup file called filename.a and an empty +file we will write to called filename.b. When close() is called on the file, if both filename.a and filename.b are valid, the original file's data is replaced with the data of filename.b. If filename.b is not valid, no changes are made. Write test applications to ensure your reference monitor behaves properly in different cases and to test attacks against your monitor. @@ -40,20 +38,21 @@ in different cases and to test attacks against your monitor. 1. Not modify or disable any functionality of any [RepyV2 API calls](../Programming/RepyV2API.md), such as: * Creating new files * Opening an existing file - * Reading valid file using readat() + * Reading valid backup using readat() * Writing to file using writeat(). This includes invalid writes, because 'S' and 'E' may later be written to the begining and end of the file respectively. -2. Check if the file starts with 'S' and ends with 'E', only when close() is called. -3. Not produce any errors +2. Check if file contents starts with 'S' and ends with 'E', only when close() is called. +3. Update the original file with the new data IF the new data is valid on close(). +4. Not produce any errors * Normal operations should not be blocked or produce any output * Invalid operations should not produce any output to the user #### The Reference Monitor Should: -1. Store two copies of the same file (filename.a and filename.b) - * One is a valid backup, and the other is written to +1. Create two copies of the same file (filename.a and filename.b) + * One is a valid backup to read from, and the other is written to 2. When an app calls ABopenfile(), the method opens the A/B files, which you should name filename.a and filename.b. -3. When the app calls readat(), all reads must be performed on the valid file -4. When the app calls writeat(), all writes must be performed on the invalid file. +3. When the app calls readat(), all reads must be performed on the valid backup. +4. When the app calls writeat(), all writes must be performed on the written to file. Three design paradigms are at work in this assignment: accuracy, @@ -61,7 +60,7 @@ efficiency, and security. * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. For example, if an app -tries to read data from a valid file, this must succeed as per normal and +tries to read data from the backup file, this must succeed as per normal and must not be blocked. All situations that are not described above *must* match that of the underlying API. @@ -70,8 +69,8 @@ so performance is not compromised. For example, keeping a complete copy of every file on disk in memory would be forbidden. * Security: The attacker should not be able to circumvent the security -layer. Hence, if the attacker can cause an invalid file to be read or can -write to a valid file, then the security is compromised, for example. +layer. For example, if the attacker can cause an invalid file to be saved, read the "write to" file, or can +write to the backup file we read from, then the security is compromised. From 477d17e81d6278e7c8aa32d4b42539cdd3a5bc78 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 22 Sep 2019 05:36:49 -0400 Subject: [PATCH 034/122] Update to match language of Part One --- EducationalAssignments/ABStoragePartTwo.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/EducationalAssignments/ABStoragePartTwo.md b/EducationalAssignments/ABStoragePartTwo.md index 1c0e3298..4f246566 100644 --- a/EducationalAssignments/ABStoragePartTwo.md +++ b/EducationalAssignments/ABStoragePartTwo.md @@ -20,11 +20,11 @@ Three design paradigms are at work in this assignment: accuracy, efficiency, and Within the context of this assignment these design paradigms translate to: - * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. For example, if an app tries to read data from a valid file, this must succeed as per normal and must not be blocked. All situations that are not described above must match that of the underlying API. + * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. For example, if an app tries to read data from the backup file, this must succeed as per normal and must not be blocked. All situations that are not described above must match that of the underlying API. * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. For example, keeping a complete copy of every file on disk in memory would be forbidden. - * Security: The attacker should not be able to circumvent the security layer. Hence, if the attacker can cause an invalid file to be read or can write to a valid file, then the security is compromised, for example. + * Security: The attacker should not be able to circumvent the security layer. For example, if the attacker can cause an invalid file to be saved, read the "write to" file, or can write to the backup file we read from, then the security is compromised. You will submit a zip file containing all of the tests you have created. You will gain points for every student's reference monitor you find a flaw in. It is good if multiple tests of yours break a student's reference monitor, but you gain the same number of tests whether one or more tests break the layer. From 3246e58d4eb94e4144c7ef69d9fc2494cefb6f93 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:15:46 -0400 Subject: [PATCH 035/122] Updated the wiki links to links to the md files Re line 10 Not sure where that link is suppose to go so I'm going to leave that one alone Re line 27 I looked at the seattle page and it redirects to build instructions so thats what I did here too and linked the page to the build instructions page I added some of the unused links regarding RePyV2 --- Programming/ProgrammersPage.md | 43 +++++++++++++++++----------------- 1 file changed, 22 insertions(+), 21 deletions(-) diff --git a/Programming/ProgrammersPage.md b/Programming/ProgrammersPage.md index 5c13f5bd..603965ba 100644 --- a/Programming/ProgrammersPage.md +++ b/Programming/ProgrammersPage.md @@ -1,65 +1,66 @@ -# Programmer Portal +# Programmer Portal This page features information and references regarding how to use write programs on Seattle. Seattle nodes run code written in a special language called Repy. Repy is a subset of the [Python language](http://www.python.org/) (version 2.5 / version 2.6) invented specifically for Seattle. -## Before You Begin +## Before You Begin Make sure you have all the necessary tools installed: * **Step 1**: Python2.5 or Python2.6 must be installed in order to use Repy. [Download Python2.6](http://www.python.org/download/releases/2.6.4/) **WINDOWS USERS:** For instructions on how to check if you already have the correct version of Python installed, and for directions on how to install Python2.6, [wiki:InstallPythonOnWindows click here] * **Step 2**: You will have to download and install repy before starting the tutorials below: **[Download repy!](https://seattleclearinghouse.poly.edu/download/flibble/)** -## Tutorials +## Tutorials You can start with several different tutorials depending on your background. - * [wiki:PythonTutorial All of the Python You Need to Understand Repy (and None You Don't)]. Start here if you're new to Python and Repy! + * [PythonTutorial All of the Python You Need to Understand Repy (and None You Don't)](PythonTutorial.md) Start here if you're new to Python and Repy! - * [wiki:PythonVsRepy All of the Python You Need to Forget to Use Repy]. Use this tutorial if you already know Python. + * [PythonVsRepy All of the Python You Need to Forget to Use Repy](PythonVsRepy.md) Use this tutorial if you already know Python. + * [All of the Python You Need to Forget to Use RepyV2](PythonVsRepyV2.md) After doing one of the above tutorials, do the following tutorial to learn how to use Repy specific features and functionality: - * [wiki:RepyTutorial Repy Tutorial]. Try this tutorial to learn more about Repy features and the language API. + * [RepyTutorial Repy Tutorial](RepyTutorial.md) Try this tutorial to learn more about Repy features and the language API. + * [RePyV2 Tutorial](RepyV2Tutorial.md) If you would prefer to use Repy V2, it is worthwhile to read through the following tutorials: - * [wiki:RepyV2CheckoutAndUnitTests Checking out Repy V2 and Running the included unit tests] + * [RepyV2CheckoutAndUnitTests Checking out Repy V2 and Running the included unit tests](../Contributing/BuildInstructions.md) + * [RepyV2SecurityLayers How to use Security Layers with Repy V2](RepyV2SecurityLayers.md) - * [wiki:RepyV2SecurityLayers How to use Security Layers with Repy V2] +## Other Resources + * An easy way to start learning Seattle is to watch our [UnderstandingSeattle/DemoVideo five-minute demo](https://seattle.poly.edu/static/demo.mov). -## Other Resources - * An easy way to start learning Seattle is to watch our [wiki:UnderstandingSeattle/DemoVideo five-minute demo]. + * Use the [RepyApi Repy API Reference](RepyApi.md) as a quick guide to Repy's API once you are comfortable with the language. - * Use the [wiki:RepyApi Repy API Reference] as a quick guide to Repy's API once you are comfortable with the language. - - * Programmers writing code for Repy V2 may want to read about the [wiki:RepyV1vsRepyV2 differences between Repy V1 and Repy V2]. + * Programmers writing code for Repy V2 may want to read about the [RepyV1vsRepyV2 differences between Repy V1 and Repy V2](RepyV1vsRepyV2.md). * There is a growing list of library code you can download and use with Repy (see [seattlelib](http://seattle.poly.edu/svn/seattle/trunk/seattlelib/) in the Seattle repository for examples). Read more about RepyHelper for the details of how to include Repy code in Python programs. - * Documentation about the Seattle Standard Library can be seen at [wiki:SeattleLib] + * Documentation about the Seattle Standard Library can be seen at [SeattleLib](SeattleLib_v1/ProgrammerResources.md) - * Continuous Integration Guide for Seattle Projects: [wiki:ContinuousIntegrationGuide] + * Continuous Integration Guide for Seattle Projects: [ContinuousIntegrationGuide](../Contributing/ContinuousIntegration.md) -## Editing Repy files -### Vim +## Editing Repy files +### Vim João Moreno has also provided a [VIM syntax file](http://www.vim.org/scripts/script.php?script_id=2546) for Repy that will syntax color Repy programs. You can also choose to automatically color Repy code with python-mode in emacs. In your .emacs file, add the following line: ``` -(add-to-list 'auto-mode-alist '(" - +(add-to-list 'auto-mode-alist '(" + .repy$" . python-mode)) ``` -### gedit +### gedit In order to have proper syntax highlighting for Repy, `repy.lang` needs to be moved into the folder that stores the defined languages. It is a slightly modified version of the python language spec. * For system-wide changes, move it to: `/usr/share/gtksourceview-3.0/language-specs/repy.lang` * For a single user, move it to: `~/.local/share/gtksourceview-3.0/language-specs/repy.lang` -You can download `repy.lang` [raw-attachment:repy.lang here] \ No newline at end of file +You can download `repy.lang` [raw-attachment:repy.lang here] From f9e7809dea96ba8263e0787ef6dd371a4caf0ac9 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:21:31 -0400 Subject: [PATCH 036/122] Added the wiki links --- .../SeattleLib_v1/openDHTadvertise.repy.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/Programming/SeattleLib_v1/openDHTadvertise.repy.md b/Programming/SeattleLib_v1/openDHTadvertise.repy.md index a438ea19..ab2ec7df 100644 --- a/Programming/SeattleLib_v1/openDHTadvertise.repy.md +++ b/Programming/SeattleLib_v1/openDHTadvertise.repy.md @@ -1,7 +1,7 @@ -# openDHTadvertise.repy +# openDHTadvertise.repy Utilizes OpenDHT to advertise availability of nodes. -### Functions +### Functions ``` openDHTadvertise_announce(key, value, ttlval, concurrentevents=5, proxiestocheck=5, timeout=None) ``` @@ -29,14 +29,14 @@ openDHTadvertise_get_proxy_list(maxnumberofattempts=5, concurrentevents=5) ``` Retrieves a list of active OpenDHT proxies. -### Example Usage +### Example Usage ...? only located in .svn directories? -### Includes -[wiki:SeattleLib/random.repy] +### Includes +[include random.repy](random.repy.md) -[wiki:include sha.repy] +[include sha.repy](sha.repy.md) -[wiki:include xmlrpc_client.repy] +[include xmlrpc_client.repy](xmlrpc_client.repy.md) -[wiki:include parallelize.repy] +[include parallelize.repy](parallelize.repy.md) From 235506fa79a886fcf23b007940c8490d7c449831 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:36:03 -0400 Subject: [PATCH 037/122] Update advertise.repy.md --- Programming/SeattleLib_v1/advertise.repy.md | 22 ++++++++++----------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/Programming/SeattleLib_v1/advertise.repy.md b/Programming/SeattleLib_v1/advertise.repy.md index 4c81b913..2b57875b 100644 --- a/Programming/SeattleLib_v1/advertise.repy.md +++ b/Programming/SeattleLib_v1/advertise.repy.md @@ -1,8 +1,8 @@ -# advertise.repy +# advertise.repy This module allows for different node to be announced via different services (central advertise service, OpenDHT, or both). -### Functions +### Functions ``` advertise_announce(key, value, ttlval, concurrentevents=2, graceperiod=10, timeout=60) ``` @@ -13,7 +13,7 @@ advertise_announce(key, value, ttlval, concurrentevents=2, graceperiod=10, timeo * concurrentevents is how many services to announce in parallel. * graceperiod and timeout are both optional parameters. -### Example Usage +### Example Usage ``` advertise_lookup(key, maxvals=100, lookuptype=None, concurrentevents=2, graceperiod=10, timeout=60) @@ -24,7 +24,7 @@ advertise_lookup(key, maxvals=100, lookuptype=None, concurrentevents=2, graceper * lookuptype defaults to look in all types. * lookuptype, concurrentevents, graceperiod, and timeout are all optional. -### Usage +### Usage ``` #retrieve node list from advertise_lookup @@ -36,15 +36,15 @@ node_list = advertise_lookup(node_state_pubkey, maxvals = 10*1024*1024, lookupty advertise_announce(advertisekey, str(my_name), adTTL) ``` -### Includes -[wiki:SeattleLib/listops.repy] +### Includes +[SeattleLib/listops.repy](listops.repy.md) -[wiki:SeattleLib/openDHTadvertise.repy] +[SeattleLib/openDHTadvertise.repy](openDHTadvertise.repy.md) -[wiki:SeattleLibcentralizedadvertise.repy] +[SeattleLibcentralizedadvertise.repy](centralizedadvertise.repy.md) -[wiki:SeattleLib/DORadvertise.repy] - -[wiki:SeattleLib/parallelize.repy] +[SeattleLib/DORadvertise.repy](DORadvertise.repy.md) +[SeattleLib/parallelize.repy](parallelize.repy.md) +[Back to NodeAdvertising](NodeAdvertising.md) From c8558bc885981268a4bacc1892764b8f02a00f7b Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:37:15 -0400 Subject: [PATCH 038/122] Update DORadvertise.repy.md --- .../SeattleLib_v1/DORadvertise.repy.md | 20 ++++++++++--------- 1 file changed, 11 insertions(+), 9 deletions(-) diff --git a/Programming/SeattleLib_v1/DORadvertise.repy.md b/Programming/SeattleLib_v1/DORadvertise.repy.md index 7ea46f52..bc6b3669 100644 --- a/Programming/SeattleLib_v1/DORadvertise.repy.md +++ b/Programming/SeattleLib_v1/DORadvertise.repy.md @@ -1,11 +1,11 @@ -# DORadvertise.repy +# DORadvertise.repy Another method of advertising nodes in the Seattle library. This module advertises to the Digital Object Registry run by CNRI. -### Functions +### Functions ``` DORadvertise_announce(key, value, ttlval, timeout=None) ``` Adds a (key, value) pair into the DOR. - Notes: + Notes: * ttlval describes the length of time in seconds the tuple exists in the DOR. * Exceptions are raised if there are errors within the XML-RPC client. @@ -15,18 +15,20 @@ DORadvertise_announce(key, value, ttlval, timeout=None) DORadvertise_lookup(key, maxvals=100, timeout=None) ``` Looks up a stored value under the key in the DOR. - Notes: + Notes: * maxvals is the maximum number of values returned. * timeout is the number of seconds spent before the process quits. -### Usage +### Usage ???couldn't find any? -### Include -[wiki:SeattleLib/sockettimeout.repy] +### Include +[wiki:SeattleLib/sockettimeout.repy](sockettimeout.repy.md) -[wiki:SeattleLib/httpretrieve.repy] +[wiki:SeattleLib/httpretrieve.repy](httpretrieve.repy.md) -[wiki:SeattleLib/xmlparse.repy] +[wiki:SeattleLib/xmlparse.repy](xmlparse.repy.md) + +[Back to NodeAdvertising](NodeAdvertising.md) From fa7f69412bb6761e8da26d19a34e4d186da48932 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:37:31 -0400 Subject: [PATCH 039/122] Update DORadvertise.repy.md --- Programming/SeattleLib_v1/DORadvertise.repy.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Programming/SeattleLib_v1/DORadvertise.repy.md b/Programming/SeattleLib_v1/DORadvertise.repy.md index bc6b3669..7e5bd86f 100644 --- a/Programming/SeattleLib_v1/DORadvertise.repy.md +++ b/Programming/SeattleLib_v1/DORadvertise.repy.md @@ -25,10 +25,10 @@ DORadvertise_lookup(key, maxvals=100, timeout=None) ???couldn't find any? ### Include -[wiki:SeattleLib/sockettimeout.repy](sockettimeout.repy.md) +[SeattleLib/sockettimeout.repy](sockettimeout.repy.md) -[wiki:SeattleLib/httpretrieve.repy](httpretrieve.repy.md) +[SeattleLib/httpretrieve.repy](httpretrieve.repy.md) -[wiki:SeattleLib/xmlparse.repy](xmlparse.repy.md) +[SeattleLib/xmlparse.repy](xmlparse.repy.md) [Back to NodeAdvertising](NodeAdvertising.md) From 65f80a7df8956de603df76140d9df1f1d5c4f4fc Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:38:16 -0400 Subject: [PATCH 040/122] Update NodeAdvertising.md --- Programming/SeattleLib_v1/NodeAdvertising.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/Programming/SeattleLib_v1/NodeAdvertising.md b/Programming/SeattleLib_v1/NodeAdvertising.md index 098f434e..b1a65586 100644 --- a/Programming/SeattleLib_v1/NodeAdvertising.md +++ b/Programming/SeattleLib_v1/NodeAdvertising.md @@ -1,13 +1,13 @@ -### Description +### Description -Seattle uses a node based service where available resources are stored by value and key pairs. All available resources (otherwise known as nodes) are thus hashed to a global store (otherwise known as advertising). The Seattle Standard Library provides three different methods of advertising nodes: [wiki:SeattleLibcentralizedadvertise.repy], [wiki:SeattleLib/openDHTadvertise.repy], or [wiki:SeattleLib/DORadvertise.repy]. +Seattle uses a node based service where available resources are stored by value and key pairs. All available resources (otherwise known as nodes) are thus hashed to a global store (otherwise known as advertising). The Seattle Standard Library provides three different methods of advertising nodes: [SeattleLibcentralizedadvertise.repy](centralizedadvertise.repy.md), [openDHTadvertise.repy](openDHTadvertise.repy.md), or [DORadvertise.repy](DORadvertise.repy.md). -[wiki:SeattleLibcentralizedadvertise.repy] uses a centralized hash table to store all the values, which runs on the main Seattle server. This may be desirable to users who do not want to depend on the OpenDHT client in case of failure, etc. +[SeattleLibcentralizedadvertise.repy](centralizedadvertise.repy.md) uses a centralized hash table to store all the values, which runs on the main Seattle server. This may be desirable to users who do not want to depend on the OpenDHT client in case of failure, etc. -[wiki:SeattleLib/openDHTadvertise.repy] uses the OpenDHT client to store key value pairs. +[openDHTadvertise.repy](openDHTadvertise.repy.md) uses the OpenDHT client to store key value pairs. -[wiki:SeattleLib/DORadvertise.repy] uses the CNRI service. +[DORadvertise.repy](DORadvertise.repy.md). uses the CNRI service. -One of these services may be chosen for exclusive use, but [wiki:SeattleLibadvertise.repy] is the most common choice, as it combines all three services and allows the user to pick a specific implementation of node advertising. +One of these services may be chosen for exclusive use, but [advertise.repy](advertise.repy.md) is the most common choice, as it combines all three services and allows the user to pick a specific implementation of node advertising. -[Back to SeattleLibWiki] \ No newline at end of file +[Back to SeattleLibWiki](../) From 9d73cc5dfbd78a5e23dc9e30bf716beaf8898b73 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:39:43 -0400 Subject: [PATCH 041/122] Update centralizedadvertise.repy.md --- .../SeattleLib_v1/centralizedadvertise.repy.md | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-) diff --git a/Programming/SeattleLib_v1/centralizedadvertise.repy.md b/Programming/SeattleLib_v1/centralizedadvertise.repy.md index 8f5cb133..cc2bc893 100644 --- a/Programming/SeattleLib_v1/centralizedadvertise.repy.md +++ b/Programming/SeattleLib_v1/centralizedadvertise.repy.md @@ -1,8 +1,8 @@ -# centralizedadvertise.repy +# centralizedadvertise.repy This module provides a hash table service for nodes. Adds and removes entries to a centralized hash table. This service runs on seattle.cs, which is also known as satya.cs. See CentralizedAdvertiseService for more details. -### Functions +### Functions @@ -24,7 +24,7 @@ centralizedadvertise_lookup(key, maxvals=100) * maxvals must be a positive integer that describes how many values to return. * Network / Timeout exception are raised if there are connection errors. -### Example Usage +### Example Usage ``` #advertize that the current client has started @@ -32,9 +32,10 @@ my_advetisement_info = getmyip() + ":"+ str(mycontext['myport']) + ":" + str(key centralizedadvertise_announce(mycontext['experiment_name'], my_advetisement_info, ADVERTISE_PERSIST) ``` -### Includes -[wiki:SeattleLib/sockettimeout.repy] -[wiki:SeattleLib/serialize.repy] +### Includes +[sockettimeout.repy](sockettimeout.repy.md) + +[serialize.repy](serialize.repy.md) From a082194e806a549028f3e97274c92ca9f973eae6 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:42:45 -0400 Subject: [PATCH 042/122] Update AdvertiseObjects.repy.md --- .../SeattleLib_v1/AdvertiseObjects.repy.md | 20 +++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/Programming/SeattleLib_v1/AdvertiseObjects.repy.md b/Programming/SeattleLib_v1/AdvertiseObjects.repy.md index 68836027..b332f504 100644 --- a/Programming/SeattleLib_v1/AdvertiseObjects.repy.md +++ b/Programming/SeattleLib_v1/AdvertiseObjects.repy.md @@ -1,19 +1,19 @@ -# AdvertiseObjects.repy +# AdvertiseObjects.repy Provides two objects which makes advertising more efficient. Can be used as an alternative to [wiki:Advertise.repy]. -LookupCache caches lookup results to increase performance in programs which calls lookup services frequently. +LookupCache caches lookup results to increase performance in programs which calls lookup services frequently. AdvertisePipe stores a list of (key, value) tuples and advertises them concurrently in a single thread. -# Functions +# Functions LookupCache ``` lookup(self, key, maxvals=100, lookuptype=['central','opendht','DOR'], concurrentevents=2, graceperiod=10, timeout=60) ``` - Exactly the same as [wiki:Advertise.repy]. - Note: + Exactly the same as [advertise.repy](advertise.repy.md). + Note: * self refers to its own cache. @@ -22,7 +22,7 @@ AdvertisePipe add(self, key, value) ``` Adds a (key, value) pair to the advertise pipe. - Note: + Note: * self refers to its own cache. * Each value added is given an unique handle, which is used in remove. @@ -31,14 +31,14 @@ add(self, key, value) remove(self, handle) ``` Removes a (key, value) pair from the advertise pipe. - Note: + Note: * handle is an unique identifier for each value in the advertise pipe. * self refers to its own cache -# Usage +# Usage no examples? -# Includes -[wiki:Advertise.repy] +# Includes +[advertise.repy](advertise.repy.md) From 1d212df3d677a466230e8d3a01c6df3038251bd7 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:43:08 -0400 Subject: [PATCH 043/122] Update NAT_advertisement.repy.md --- .../SeattleLib_v1/NAT_advertisement.repy.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/Programming/SeattleLib_v1/NAT_advertisement.repy.md b/Programming/SeattleLib_v1/NAT_advertisement.repy.md index 9c611b4f..e6c424d3 100644 --- a/Programming/SeattleLib_v1/NAT_advertisement.repy.md +++ b/Programming/SeattleLib_v1/NAT_advertisement.repy.md @@ -1,8 +1,8 @@ -# NAT_advertisement.repy +# NAT_advertisement.repy -Abstracts the task of looking up and advertising servers and forwarders. Allows for those using NAT_layer to advertise. Utilizes [wiki:Advertise.repy] to achieve this. +Abstracts the task of looking up and advertising servers and forwarders. Allows for those using NAT_layer to advertise. Utilizes [advertise.repy](advertise.repy.md) to achieve this. -### Functions +### Functions ``` nat_forwarder_advertise(ip, serverport, clientport) @@ -33,12 +33,12 @@ nat_server_list_lookup(key) nat_toggle_advertisement(enabled, threadRun=True) ``` Toggles the state of the advertisement. - Notes: + Notes: * threadRun controls the advertisement thread. -### Usage +### Usage ??? -### Includes -[wiki:Advertise.repy] \ No newline at end of file +### Includes +[advertise.repy](advertise.repy.md) From 2736ae9f0ffd77e92486273db6138055b1d641f8 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:45:46 -0400 Subject: [PATCH 044/122] Update Cryptography.md --- Programming/SeattleLib_v1/Cryptography.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Programming/SeattleLib_v1/Cryptography.md b/Programming/SeattleLib_v1/Cryptography.md index 17e1bd9b..24238266 100644 --- a/Programming/SeattleLib_v1/Cryptography.md +++ b/Programming/SeattleLib_v1/Cryptography.md @@ -1,4 +1,4 @@ -### Description +### Description Seattle supports various cryptography choices. Since Seattle includes a lot of network communication, encrypting data is very important. This prevents 3rd party interference. @@ -6,4 +6,4 @@ Many of the modules in this category are ported directly from Python equivalents Note also a few modules are very experimental in nature and may not have full functionality. All of these discrepancies will be noted as well per section. -[Back to SeattleLibWiki] \ No newline at end of file +[Back to SeattleLibWiki](../) From b4028198d2f3ca798b66432b550f6a32892c14a0 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:46:24 -0400 Subject: [PATCH 045/122] Update Time.md --- Programming/SeattleLib_v1/Time.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/Programming/SeattleLib_v1/Time.md b/Programming/SeattleLib_v1/Time.md index a86e0001..3572cca3 100644 --- a/Programming/SeattleLib_v1/Time.md +++ b/Programming/SeattleLib_v1/Time.md @@ -1,7 +1,9 @@ -### Description +### Description Like many services, Seattle contains its own time module, which provides various time related functions like getting the time and updating the time. Seattle provides both TCP and NTP time services. TCP, otherwise known as Transmission Control Protocol, utilizes timestamps to keep track of time. NTP, or Network Time Protocol, synchronizes clocks on different machines by using various jitter buffers. -In any case, all programmers should include [wiki:SeattleLib/time.repy], which ties both services into one. \ No newline at end of file +In any case, all programmers should include [time.repy](time.repy.md), which ties both services into one. + +[Back to SeattleLibWiki](../) From 41cd9ef5c04791dbe96c51e9464352cee667db46 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:49:37 -0400 Subject: [PATCH 046/122] Update time.repy.md --- Programming/SeattleLib_v1/time.repy.md | 10 ++++++---- 1 file changed, 6 insertions(+), 4 deletions(-) diff --git a/Programming/SeattleLib_v1/time.repy.md b/Programming/SeattleLib_v1/time.repy.md index ebc196d2..5eda2b0e 100644 --- a/Programming/SeattleLib_v1/time.repy.md +++ b/Programming/SeattleLib_v1/time.repy.md @@ -1,9 +1,11 @@ -# time.repy +# time.repy A include file to tie the time interface together. Contains no functions. -### Includes +### Includes -[wiki:SeattleLib/ntp_time.repy] +[wiki:SeattleLib/ntp_time.repy](ntp_time.repy.md) -[wiki:SeattleLib/tcp_time.repy] \ No newline at end of file +[wiki:SeattleLib/tcp_time.repy](tcp_time.repy.md) + +[Back to Time](Time.md) From 870aad8eb4c1e2bfa6812f5799ad7c222d0899f2 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:53:27 -0400 Subject: [PATCH 047/122] Update rsa.repy.md --- Programming/SeattleLib_v1/rsa.repy.md | 33 ++++++++++++++------------- 1 file changed, 17 insertions(+), 16 deletions(-) diff --git a/Programming/SeattleLib_v1/rsa.repy.md b/Programming/SeattleLib_v1/rsa.repy.md index 0f7ca28d..8486463c 100644 --- a/Programming/SeattleLib_v1/rsa.repy.md +++ b/Programming/SeattleLib_v1/rsa.repy.md @@ -1,11 +1,11 @@ -# rsa.repy +# rsa.repy This is the main interface for using the ported RSA implementation. See http://en.wikipedia.org/wiki/RSA for more details. This port to repy is primarily taken from Dwayne C. Litzenberger's (www.dlitz.net) python code. -### Functions +### Functions ``` @@ -14,10 +14,10 @@ def rsa_gen_pubpriv_keys(bitsize): Will generate a new key object with a key size of the argument bitsize and return it. A recommended value would be 1024. - Notes: + Notes: * bitsize is the number of bits that the key should have. This means the modulus (publickey - n) will be in the range [2**(bitsize - 1), 2**(bitsize) - 1] - * Will return a key object that rsa_encrypt, rsa_decrypt, rsa_sign, and rsa_validate can use to preform their tasts. + * Will return a key object that rsa_encrypt, rsa_decrypt, rsa_sign, and rsa_validate can use to preform their tasts. @@ -29,7 +29,7 @@ def rsa_encrypt(message, publickey): Will use the key to encrypt the message string. If the string is to large to be encrypted it will be broken into chunks that are suitable for the keys modulus by the _rsa_chopstring function. - Notes: + Notes: * message is the string to be encrypted, there is no restriction on size. * publickey must be a valid publickey dictionary of the form {'n': 1.., 'e': 6..} with the keys being 'n' and 'e'. @@ -49,7 +49,7 @@ def rsa_decrypt(cypher, privatekey): If the plaintext string was to large to be encrypted it will use _rsa_gluechops and _rsa_unpicklechops to reassemble the origional plaintext after the individual peices are decrypted. - Notes: + Notes: * cypher is the encrypted string that was returned by rsa_encrypt. Example: " 142247030 31373650 44827705" * privatekey must be a valid privatekey dictionary of the form {'d':1.., 'p':1.. 'q': 1..} with the keys being 'd', 'p', and 'q'. @@ -67,7 +67,7 @@ def rsa_sign(message, privatekey): Will use the key to sign the plaintext string. - Notes: + Notes: * message is the string to be encrypted, there is no restriction on size. * publickey must be a valid publickey dictionary of the form {'n': 1.., 'e': 6..} with the keys being 'n' and 'e'. @@ -85,7 +85,7 @@ def rsa_verify(cypher, publickey): Will use the private key to decrypt the cypher string. - Notes: + Notes: * If the plaintext string was to large to be encrypted it will use _rsa_gluechops and _rsa_unpicklechops to reassemble the origional plaintext after the individual peices are decrypted. * cypher is the encrypted string that was returned by rsa_encrypt. Example: " 142247030 31373650 44827705" @@ -103,7 +103,7 @@ def rsa_is_valid_privatekey(key): This tries to determine if a key is valid. If it returns False, the key is definitely invalid. If True, the key is almost certainly valid. - Notes: + Notes: * key is a dictionary of the form {'d':1.., 'p':1.. 'q': 1..} with the keys 'd', 'p', and 'q' * If the key is valid, True will be returned. Otherwise False will be returned. @@ -125,7 +125,7 @@ def rsa_publickey_to_string(publickey): ``` To convert a publickey to a string. It will read the publickey which should a dictionary, and return it in the appropriate string format. - Notes: + Notes: * Must be a valid publickey dictionary of the form {'n': 1.., 'e': 6..} with the keys 'n' and 'e'. * Raises ValueError if the publickey is invalid. @@ -140,7 +140,7 @@ def rsa_string_to_publickey(mystr): To read a private key string and return a dictionary in the appropriate format: {'n': 1.., 'e': 6..} with the keys 'n' and 'e'. - Notes: + Notes: * mystr is a string containing the publickey, should be in the format created by the function rsa_publickey_to_string. Example if e=3 and n=21, mystr = "3 21" * Raises ValueError if the string containing the privateky is in a invalid format. @@ -173,7 +173,7 @@ def rsa_privatekey_to_file(key,filename): To write a privatekey to a file. It will convert the privatekey which should a dictionary, to the appropriate format and write it to a file, so that it can be read by rsa_file_to_privatekey. - Notes: + Notes: * privatekey must be a valid privatekey dictionary of the form {'d':1.., 'p':1.. 'q': 1..} with the keys 'd', 'p', and 'q' * filename is the string containing the name for the desired publickey file. @@ -189,7 +189,7 @@ def rsa_file_to_privatekey(filename): To read a file containing a key that was created with rsa_privatekey_to_file and return it in the appropriate format: {'d':1.., 'p':1.. 'q': 1..} with the keys 'd', 'p', and 'q'. - Notes: + Notes: * filename is the name of the file containing the privatekey. * Raises ValueError if the file contains an invalid private key string. @@ -217,8 +217,9 @@ def rsa_matching_keys(privatekey, publickey): ``` -### Includes +### Includes -[wiki:SeattleLib/random.repy] -[wiki:SeattleLib/pycryptorsa.repy] +[random.repy](random.repy.md) + +[pycryptorsa.repy](pycryptorsa.repy.md) From 3d6b612ce482813c28d27b11ddf5f047defc6eac Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:54:42 -0400 Subject: [PATCH 048/122] Update sshkey.repy.md --- Programming/SeattleLib_v1/sshkey.repy.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/Programming/SeattleLib_v1/sshkey.repy.md b/Programming/SeattleLib_v1/sshkey.repy.md index 74aff69d..adbcb3a1 100644 --- a/Programming/SeattleLib_v1/sshkey.repy.md +++ b/Programming/SeattleLib_v1/sshkey.repy.md @@ -1,10 +1,10 @@ -# sshkey.repy +# sshkey.repy sshkey.repy provides an utility for reading public and private keys. This module is used whenever a sshkey needs to be decrypted, and helps the client by providing a level of abstraction in reading these keys. This module is capable of reading private keys encrypted with DES3 as well, in addition to the keys encrypted by [wiki:SeattleLib/rsa.repy]. -### Functions +### Functions ``` def close(self): @@ -17,7 +17,7 @@ def read(self, n = -1): ``` Read at most size bytes from the file (less if the read hits EOF before obtaining size bytes). - Notes: + Notes: * If the size argument is negative or omitted, read all data until EOF is reached. The bytes are returned as a string object. An empty string is returned when EOF is encountered immediately. * Raises ValueError if I/O operation on closed file. @@ -29,7 +29,7 @@ def sshkey_file_to_privatekey(filename, password=None): ``` Reads a ssh private key file and returns the key in a format suitable to used by rsa.repy. - Notes: + Notes: * filename is the name of the file containing the ssh-rsa private key. Key should be either unencrypted or encrypted with DES3. * password is the password used to encrypt the ssh-rsa private key. @@ -51,7 +51,7 @@ def sshkey_file_to_publickey(filename): ``` Reads a ssh public key file and returns the key in a format suitable to used by rsa.repy. - Notes: + Notes: * filename is the name of the file containing the ssh-rsa publickey. * Raises sshkey_SSHException if private key was unable to be decoded. This could happen for any number of reasons and the only quick fix is to generate a new key that is supported. This could happen if the file contains a dss key. @@ -60,7 +60,7 @@ def sshkey_file_to_publickey(filename): * Returns a publickey dictionary in the format used by the rsa.repy module: {'n': 1.., 'e': 6..} with the keys 'n' and 'e'. -### Usage +### Usage ``` @@ -98,6 +98,6 @@ def sshkey_file_to_publickey(filename): assert(publickey['e'] == e) ``` -### Includes +### Includes -[wiki:SeattleLib/sshkey_paramiko.repy] \ No newline at end of file +[sshkey_paramiko.repy](sshkey_paramiko.repy.md) From 88d049aa5b655f3c559202fe6619735f2248abfe Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:55:52 -0400 Subject: [PATCH 049/122] Update signeddata.repy.md --- Programming/SeattleLib_v1/signeddata.repy.md | 26 ++++++++++---------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/Programming/SeattleLib_v1/signeddata.repy.md b/Programming/SeattleLib_v1/signeddata.repy.md index 9ecaf2d2..da73d74b 100644 --- a/Programming/SeattleLib_v1/signeddata.repy.md +++ b/Programming/SeattleLib_v1/signeddata.repy.md @@ -1,20 +1,20 @@ -# signeddata.repy +# signeddata.repy This modules allows data to be signed in order to prevent attacks. By signing the data, we can create signatures, which greatly increase security since data signatures can then be checked at the start and at the end of the transfer. We are trying to prevent four main types of attacks. They are described below. -**Replay attack**: When someone provides information you signed before to try to get you to perform an old action again. For example, A sends messages to the node manager to provide a VM to B (B intercepts this traffic). Later A acquires the VM again. B should not be able to replay the messages A sent to the node manager to have the VM transferred to B again. +**Replay attack**: When someone provides information you signed before to try to get you to perform an old action again. For example, A sends messages to the node manager to provide a VM to B (B intercepts this traffic). Later A acquires the VM again. B should not be able to replay the messages A sent to the node manager to have the VM transferred to B again. -**Freeze attack**: When an attacker can act as a man-in-the-middle and provide stale information to an attacker. For example, B can intercept all traffic between the node manager and A. If C makes a change on the node manager, then B should not be able to prevent A from seeing the change (at least within some time bound). +**Freeze attack**: When an attacker can act as a man-in-the-middle and provide stale information to an attacker. For example, B can intercept all traffic between the node manager and A. If C makes a change on the node manager, then B should not be able to prevent A from seeing the change (at least within some time bound). -**Out of sequence attack**: When someone can skip sending some messages but deliver others. For example, A wants to stop the current program, upload a new copy of the program, and start the program again. It should be possible for A to specify that these actions must be performed in order and without skipping any of the prior actions (regardless of failures, etc.). +**Out of sequence attack**: When someone can skip sending some messages but deliver others. For example, A wants to stop the current program, upload a new copy of the program, and start the program again. It should be possible for A to specify that these actions must be performed in order and without skipping any of the prior actions (regardless of failures, etc.). -**Misdelivery attack**: Messages should only be acted upon by the nodes that the user intended. A malicious party should not be able to "misdeliver" a message and have a different node perform the action. +**Misdelivery attack**: Messages should only be acted upon by the nodes that the user intended. A malicious party should not be able to "misdeliver" a message and have a different node perform the action. -### Functions +### Functions ``` def signeddata_is_valid_expirationtime(expirationtime): @@ -26,14 +26,14 @@ def signeddata_is_valid_timestamp(timestamp): ``` def signeddata_is_valid_sequencenumber(sequencenumber): ``` - Checks if the sequence numbers are valid. They must be in the form 'tag:num' where tag doesn't contain ':',' + Checks if the sequence numbers are valid. They must be in the form 'tag:num' where tag doesn't contain ':',' n', or '!' # and num is a number. ``` def signeddata_is_valid_destination(destination): ``` - Destination is an "opaque string" or None. Should not contain a '!' or ' + Destination is an "opaque string" or None. Should not contain a '!' or ' n'. ``` @@ -83,7 +83,7 @@ def signeddata_shouldtrust(oldsigneddata, newsigneddata, publickey=None, oldsign Checks to see if the data itself can be trusted. -### Usage +### Usage ``` # get some signed data @@ -110,11 +110,11 @@ def signeddata_shouldtrust(oldsigneddata, newsigneddata, publickey=None, oldsign ``` -### Includes +### Includes -[wiki:SeattleLib/sha.repy] +[wiki:SeattleLib/sha.repy](sha.repy.md) -[wiki:SeattleLib/rsa.repy] +[wiki:SeattleLib/rsa.repy](rsa.repy.md) -[wiki:SeattleLib/time.repy] +[wiki:SeattleLib/time.repy](time.repy.md) From 7af7322255dae5ecc5f8a79c130dcf730900508a Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Sun, 13 Oct 2019 23:57:51 -0400 Subject: [PATCH 050/122] Update DataEncoding.md --- Programming/SeattleLib_v1/DataEncoding.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Programming/SeattleLib_v1/DataEncoding.md b/Programming/SeattleLib_v1/DataEncoding.md index 7d2e9c48..b44a3101 100644 --- a/Programming/SeattleLib_v1/DataEncoding.md +++ b/Programming/SeattleLib_v1/DataEncoding.md @@ -1,5 +1,5 @@ -### Description +### Description These services are used to protect transmission of data within the network. Encryption is also necessary in order to allow for serialization of the data, which is essential for data transmission as well. Programmers should take care to encrypt all their packets. -The Seattle Standard Library supports two different types of encoding methods: binary to ascii ([wiki:SeattleLibbinascii.repy]) and base64 ([wiki:SeattleLibbase64.repy]). The serialization module ([wiki:SeattleLib/serialize.repy]) allows for serialization and deserialization of various data types. \ No newline at end of file +The Seattle Standard Library supports two different types of encoding methods: binary to ascii ([binascii.repy](binascii.repy.md)) and base64 ([base64.repy](base64.repy.md)). The serialization module ([serialize.repy](serialize.repy.md)) allows for serialization and deserialization of various data types. From c4cf0d78a55c007f270854e76bb5e568748b39dd Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:00:33 -0400 Subject: [PATCH 051/122] Update ConcurrencyAndParallelism.md --- Programming/SeattleLib_v1/ConcurrencyAndParallelism.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Programming/SeattleLib_v1/ConcurrencyAndParallelism.md b/Programming/SeattleLib_v1/ConcurrencyAndParallelism.md index 6fe332a5..36e34607 100644 --- a/Programming/SeattleLib_v1/ConcurrencyAndParallelism.md +++ b/Programming/SeattleLib_v1/ConcurrencyAndParallelism.md @@ -1,7 +1,7 @@ -### Description +### Description The parallelism services offered by Seattle are purely for the client's needs and not required in any way. Running parallel processes are more efficient and may be of interest to some users, but adds a certain amount of overhead since locking must be implemented to prevent crashes and errors in critical sections of code. For this reason, libraries like [wiki:SeattleLib/semaphore.repy] and [wiki:SeattleLib/cv.repy] to help the client implement locks. Many modules here also have Python equivalents. These are linked appropriately. -[Back to SeattleLibWiki] \ No newline at end of file +[Back to SeattleLibWiki](../) From 3d4ef97fbcdb3e52c2aed01473850d21be9e1f76 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:01:12 -0400 Subject: [PATCH 052/122] Update UrlParsingAndXml.md --- Programming/SeattleLib_v1/UrlParsingAndXml.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Programming/SeattleLib_v1/UrlParsingAndXml.md b/Programming/SeattleLib_v1/UrlParsingAndXml.md index a842d3f3..95356d9f 100644 --- a/Programming/SeattleLib_v1/UrlParsingAndXml.md +++ b/Programming/SeattleLib_v1/UrlParsingAndXml.md @@ -1,9 +1,9 @@ -### Description +### Description This class of modules supported by the Seattle Standard Library deal with URLs and XML. In most cases, Seattle utilizes the XML-RPC protocol to communicate between computers. The remote procedure calls (RPC) are achieved by using HTTP requests. In XML-RPC, the parameter for the the HTTP requests can be nested, in this case with XML. -In any case that the XML-RPC service is desirable, [wiki:SeattleLib/xmlrpc_server.repy] should be used. There rest of the modules described here which are mostly helper modules to [wiki:SeattleLib/xmlrpc_server.repy]. +In any case that the XML-RPC service is desirable, [xmlrpc_server.repy](xmlrpc_server.repy.md) should be used. There rest of the modules described here which are mostly helper modules to [xmlrpc_server.repy](xmlrpc_server.repy.md). Note also there are many parallels between these modules and equivalent Python modules. Please email [shurui@cs.washington.edu] if I have failed to link any. -[Back to SeattleLibWiki] \ No newline at end of file +[Back to SeattleLibWiki](../) From 493f6b68de620a53a6ad77b38688c4623a9333d1 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:02:15 -0400 Subject: [PATCH 053/122] Update xmlrpc_common.repy.md --- .../SeattleLib_v1/xmlrpc_common.repy.md | 22 +++++++++---------- 1 file changed, 11 insertions(+), 11 deletions(-) diff --git a/Programming/SeattleLib_v1/xmlrpc_common.repy.md b/Programming/SeattleLib_v1/xmlrpc_common.repy.md index 41cd7374..c87e67df 100644 --- a/Programming/SeattleLib_v1/xmlrpc_common.repy.md +++ b/Programming/SeattleLib_v1/xmlrpc_common.repy.md @@ -1,4 +1,4 @@ -# xmlrpc_common.repy +# xmlrpc_common.repy This module helps to implement the client-side XML-RPC protocol. See http://en.wikipedia.org/wiki/XML-RPC for more details. @@ -6,13 +6,13 @@ A programmer may use this module to initiate a communication between the client Note that this module is primarily used by [wiki:SeattleLib/xmlrpc_client.repy], which programmers should include instead of this file. This file contains many functions which allow the XML-RPC protocol to be used in Python and is primary used to build the strings necessary to implement XML-RPC. -### Functions +### Functions ``` def xmlrpc_common_call2xml(method_name, params): ``` Build a XML-RPC method call to send to a XML-RPC server. - Notes: + Notes: * params is the sequence type of XML-RPC parameters. A dictionary may also be passed here, but the keys are ignored. * Returns the XML-RPC method call string. @@ -22,7 +22,7 @@ def xmlrpc_common_response2xml(param): ``` Build a XML-RPC method response to send to a XML-RPC client. This is the XML document that represents the return values or fault from a XML-RPC call. - Notes: + Notes: * param is the value that is returned (aka the response). * The XML-RPC method response string. @@ -32,7 +32,7 @@ def xmlrpc_common_fault2xml(message, code): ``` Build a XML-RPC fault response to send to a XML-RPC client. A fault response can occur from a server failure, an incorrectly generated XML request, or bad program arguments. - Notes: + Notes: * message is the string describing the fault. * code is the integer code associated with the fault. @@ -43,7 +43,7 @@ def xmlrpc_common_call2python(xml): ``` Convert a XML-RPC method call to its Python equivalent. The request from a XML-RPC client is parsed into native Python types so that the server may use the data to execute a method, as appropriate. - Notes: + Notes: * xml is the XML-RPC string to be convert. * Raises xmlrpc_common_XMLParseError on a XML-RPC structural parse error. @@ -55,21 +55,21 @@ def xmlrpc_common_response2python(xml): ``` Convert a XML-RPC method response to its Python equivalent. The response from a XML-RPC server is parsed into native Python types so that the client may use the data as appropriate. - Notes: + Notes: * xml is the XML-RPC string to be convert. * Raises xmlrpc_common_XMLParseError on a XML-RPC structural parse error. * Raises xmlparse_XMLParseError on other XML parse errors. * Returns the method results or a xmlrpc_common_Fault on reading a fault. -### Example Usage +### Example Usage ``` client = xmlrpc_client_Client("http://phpxmlrpc.sourceforge.net/server.php") print client.send_request("examples.getStateName", (1,)) ``` -### Includes -[wiki:SeattleLibbase64.repy] +### Includes +[base64.repy](base64.repy.md) -[wiki:SeattleLib/xmlparse.repy] +[xmlparse.repy](xmlparse.repy.md) From 0cb13c7a9da6912f0201d328bc7eed89ef1907d6 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:03:41 -0400 Subject: [PATCH 054/122] Update semaphore.repy.md --- Programming/SeattleLib_v1/semaphore.repy.md | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/Programming/SeattleLib_v1/semaphore.repy.md b/Programming/SeattleLib_v1/semaphore.repy.md index 6b90de90..ad92f318 100644 --- a/Programming/SeattleLib_v1/semaphore.repy.md +++ b/Programming/SeattleLib_v1/semaphore.repy.md @@ -1,8 +1,8 @@ -# semaphore.repy +# semaphore.repy Although repy already contains a locking method, this module provides a level of abstraction by giving the end users the possibility of using semaphores. Though semaphores are slightly superior to locks, this module is not required to run anything. Using this purely at the programmer's discretion and need. -### Functions +### Functions ``` def semaphore_create(): @@ -15,7 +15,7 @@ def semaphore_destroy(semaphorehandle): ``` Clean up a semaphore that is no longer needed. All currently blocked threads will be unblocked. All future uses of the semaphore will fail. - Notes: + Notes: * semaphorehandle is the semaphore handle to destroy * Returns True if it cleaned up the semaphore handle, False if the handle was already cleaned up. @@ -26,7 +26,7 @@ def semaphore_up(semaphorehandle): ``` Increment a sempahore (possibly unblocking a thread) - Notes: + Notes: * semaphorehandle is the semaphore handle generated by the create method. * Raises ValueError if the semaphorehandle is invalid. @@ -37,11 +37,11 @@ def semaphore_down(semaphorehandle): ``` Decrement a sempahore (possibly blocking this thread) - Notes: + Notes: * semaphorehandle is the semaphore handle generated by the create method. * Raises ValueError if the semaphorehandle is invalid. -### Includes +### Includes -[wiki:SeattleLib/uniqueid.repy] \ No newline at end of file +[uniqueid.repy](uniqueid.repy.md) From 8168d5a1a6e910f1931f48e108c65c348c06b414 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:04:07 -0400 Subject: [PATCH 055/122] Update uniqueid.repy.md --- Programming/SeattleLib_v1/uniqueid.repy.md | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/Programming/SeattleLib_v1/uniqueid.repy.md b/Programming/SeattleLib_v1/uniqueid.repy.md index 66d534b1..8287c7af 100644 --- a/Programming/SeattleLib_v1/uniqueid.repy.md +++ b/Programming/SeattleLib_v1/uniqueid.repy.md @@ -1,20 +1,20 @@ -# uniqueid.repy +# uniqueid.repy This is a simple module which provides an unique integer id for each function call. This exists to reduce redundancy in other libraries. -This service can be utilized directly, but is used in contexts like [wiki:SeattleLib/parallelize.repy]. +This service can be utilized directly, but is used in contexts like [parallelize.repy](parallelize.repy.md). NOTE: This will give unique ids PER FILE. If you have multiple python modules that include this, they will have the potential to generate the same ID. -### Functions +### Functions ``` def uniqueid_getid(): ``` Return a unique ID in a threadsafe way -### Usage +### Usage ``` request_id = uniqueid_getid(); -``` \ No newline at end of file +``` From 8bc5a82d3cb2b2806eee75c906db0585d5ce1a31 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:07:12 -0400 Subject: [PATCH 056/122] Update DataRetrieval.md --- Programming/SeattleLib_v1/DataRetrieval.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Programming/SeattleLib_v1/DataRetrieval.md b/Programming/SeattleLib_v1/DataRetrieval.md index d0d0a411..00826480 100644 --- a/Programming/SeattleLib_v1/DataRetrieval.md +++ b/Programming/SeattleLib_v1/DataRetrieval.md @@ -1,7 +1,7 @@ -### Description +### Description -All the modules in category deal with data retrieval of some form. Since Seattle is a P2P based client, its often necessary to know about a specific node: where it is, what is its IP address, etc. In addition to [wiki:SeattleLib/domainnameinfo.repy], which simply returns the country of origin, Seattle also provides the [wiki:SeattleLib/geoip_client.repy], which returns specific location information about the client, etc. +All the modules in category deal with data retrieval of some form. Since Seattle is a P2P based client, its often necessary to know about a specific node: where it is, what is its IP address, etc. In addition to [domainnameinfo.repy](domainnameinfo.repy.md), which simply returns the country of origin, Seattle also provides the [geoip_client.repy](geoip_client.repy.md), which returns specific location information about the client, etc. -Seattle also provides a basic abstraction to the HTTP protocol, see [wiki:SeattleLib/httpserver.repy]. +Seattle also provides a basic abstraction to the HTTP protocol, see [httpserver.repy](httpserver.repy.md). -[Back to SeattleLibWiki] \ No newline at end of file +[Back to SeattleLibWiki](../) From 4bce8dbfd534c3874d63180753e0f83390483cd6 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:10:02 -0400 Subject: [PATCH 057/122] Update NetworkCommunication.md --- Programming/SeattleLib_v1/NetworkCommunication.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/Programming/SeattleLib_v1/NetworkCommunication.md b/Programming/SeattleLib_v1/NetworkCommunication.md index 0b8000de..40608913 100644 --- a/Programming/SeattleLib_v1/NetworkCommunication.md +++ b/Programming/SeattleLib_v1/NetworkCommunication.md @@ -1,7 +1,7 @@ -### Description +### Description -This section describes the all important tools necessary for dealing with sockets. For instance, we can use the modules in the section to find out critical information about VMs, which are elements that computers whose resources you can access ([wiki:SeattleLib/getvesselsresources.repy]). We also have basic control over socket connections such as forcing hanging connections to quit ([wiki:SeattleLib/sockettimeout.repy]). +This section describes the all important tools necessary for dealing with sockets. For instance, we can use the modules in the section to find out critical information about VMs, which are elements that computers whose resources you can access ([getvesselsresources.repy](getvesselsresources.repy.md)). We also have basic control over socket connections such as forcing hanging connections to quit ([sockettimeout.repy](sockettimeout.repy.md)). -[wiki:SeattleLib/nmclient.repy] is a also included here. It contains the backend to what could be an very useful external node manager, but by itself is not fully functional. +[nmclient.repy](nmclient.repy.md) is a also included here. It contains the backend to what could be an very useful external node manager, but by itself is not fully functional. -[Back to SeattleLibWiki] \ No newline at end of file +[Back to SeattleLibWiki](../) From 8010dc44a89dc1b22156cf379bc994361d2058d0 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:15:01 -0400 Subject: [PATCH 058/122] Update ProgrammerResources.md --- Programming/SeattleLib_v1/ProgrammerResources.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/Programming/SeattleLib_v1/ProgrammerResources.md b/Programming/SeattleLib_v1/ProgrammerResources.md index d92c4af8..8bc3dfbb 100644 --- a/Programming/SeattleLib_v1/ProgrammerResources.md +++ b/Programming/SeattleLib_v1/ProgrammerResources.md @@ -1,13 +1,13 @@ -### Description +### Description -This section contains all the extraneous data structures that Seattle supports, in addition to all the various backend modules that makes seash work. There are several utilities which can help with basic programming tasks such as [wiki:SeattleLib/math.repy], [wiki:SeattleLib/random.repy], and etc. Notably modules include: +This section contains all the extraneous data structures that Seattle supports, in addition to all the various backend modules that makes seash work. There are several utilities which can help with basic programming tasks such as [wiki:SeattleLib/math.repy](math.repy.md), [random.repy](random.repy.md), and etc. Notably modules include: -[wiki:SeattleLib/safe_eval.repy] allows one to safely evaluate strings, free from the context of whatever it is in. +[safe_eval.repy](safe_eval.repy.md) allows one to safely evaluate strings, free from the context of whatever it is in. -[wiki:SeattleLibargparse.repy] checks command line arguments and separates in a way that is usable. This utility can be used within the context of whatever program it's in, making this a very useful module. +[argparse.repy](argparse.repy.md) checks command line arguments and separates in a way that is usable. This utility can be used within the context of whatever program it's in, making this a very useful module. -[wiki:SeattleLib/urlparse.repy] parses urls for network communication purposes. This is primarily used in the XML parsing section. +[urlparse.repy](urlparse.repy.md) parses urls for network communication purposes. This is primarily used in the XML parsing section. Please note that the The backend modules that are located here are mostly ones that the average user would not have to consider. -[Back to SeattleLibWiki] \ No newline at end of file +[Back to SeattleLibWiki](../) From a6c6f016fed8a36623fcd67ab7fdaf6f3455982d Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:15:12 -0400 Subject: [PATCH 059/122] Update ProgrammerResources.md --- Programming/SeattleLib_v1/ProgrammerResources.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Programming/SeattleLib_v1/ProgrammerResources.md b/Programming/SeattleLib_v1/ProgrammerResources.md index 8bc3dfbb..01ad31f5 100644 --- a/Programming/SeattleLib_v1/ProgrammerResources.md +++ b/Programming/SeattleLib_v1/ProgrammerResources.md @@ -1,6 +1,6 @@ ### Description -This section contains all the extraneous data structures that Seattle supports, in addition to all the various backend modules that makes seash work. There are several utilities which can help with basic programming tasks such as [wiki:SeattleLib/math.repy](math.repy.md), [random.repy](random.repy.md), and etc. Notably modules include: +This section contains all the extraneous data structures that Seattle supports, in addition to all the various backend modules that makes seash work. There are several utilities which can help with basic programming tasks such as [math.repy](math.repy.md), [random.repy](random.repy.md), and etc. Notably modules include: [safe_eval.repy](safe_eval.repy.md) allows one to safely evaluate strings, free from the context of whatever it is in. From 16a0c5ff96f9727f0cc3c2a03e003173aea7db21 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:18:21 -0400 Subject: [PATCH 060/122] Update SeattleLib.md Made categories as headings deserialize.repy not sure where this points to Multiplexer.repy not done not sure where this points to --- Programming/SeattleLib_v1/SeattleLib.md | 130 ++++++++++++------------ 1 file changed, 65 insertions(+), 65 deletions(-) diff --git a/Programming/SeattleLib_v1/SeattleLib.md b/Programming/SeattleLib_v1/SeattleLib.md index 6d41afa8..1ff825b5 100644 --- a/Programming/SeattleLib_v1/SeattleLib.md +++ b/Programming/SeattleLib_v1/SeattleLib.md @@ -1,80 +1,80 @@ -# Seattle Standard Library (SeattleLib) +# Seattle Standard Library (SeattleLib) This pages contains documentation about the Seattle standard library (called "SeattleLib"). The libraries are broken up by similar category for convenience. Click on each category for overarching descriptions and more information. The bolded links are modules are meant for direct usage by the client. They often aggregate multiple services. - * [wiki:SeattleLib/NodeAdvertising Node Advertising] (click for a description) - * **[wiki:SeattleLib/advertise.repy advertise.repy]** - * [wiki:SeattleLib/AdvertiseObjects.repy AdvertiseObjects.repy] - * [wiki:SeattleLib/centralizedadvertise.repy centralizedadvertise.repy] - * [wiki:SeattleLib/DORadvertise.repy DORadvertise.repy] - * [wiki:SeattleLib/NAT_advertisement.repy NAT_advertisement.repy] - * [wiki:SeattleLib/openDHTadvertise.repy openDHTadvertise.repy] + ## [Node Advertising](NodeAdvertising.md) (click for a description) + * **[advertise.repy](advertise.repy.md)** + * [AdvertiseObjects.repy](AdvertiseObjects.repy.md) + * [centralizedadvertise.repy](centralizedadvertise.repy.md) + * [DORadvertise.repy](DORadvertise.repy.md) + * [NAT_advertisement.repy](NAT_advertisement.repy.md) + * [openDHTadvertise.repy](openDHTadvertise.repy.md) - * [wiki:SeattleLib/Cryptography Cryptography] (click for a description) - * [wiki:SeattleLib/md5py.repy md5py.repy] - * [wiki:SeattleLib/pycryptorsa.repy pycryptorsa.repy] - * [wiki:SeattleLib/pyDes.repy pyDes.repy] - * [wiki:SeattleLib/rsa.repy rsa.repy] - * [wiki:SeattleLib/sha.repy sha.repy] - * **[wiki:SeattleLib/sshkey.repy sshkey.repy]** - * **[wiki:SeattleLib/sshkey_paramiko.repy sshkey_paramiko.repy]** - * [wiki:SeattleLib/signeddata.repy signeddata.repy] + ## [Cryptography](Cryptography.md) (click for a description) + * [md5py.repy](md5py.repy.md) + * [pycryptorsa.repy](pycryptorsa.repy.md) + * [wpyDes.repy](pyDes.repy.md) + * [rsa.repy](rsa.repy.md) + * [sha.repy](sha.repy.md) + * **[sshkey.repy](sshkey.repy.md)** + * **[sshkey_paramiko.repy](sshkey_paramiko.repy.md)** + * [signeddata.repy](signeddata.repy.md) - * [wiki:SeattleLib/Time Time] (click for a description) - * [wiki:SeattleLib/ntp_time.repy ntp_time.repy] - * [wiki:SeattleLib/tcp_time.repy tcp_time.repy] - * **[wiki:SeattleLib/time.repy time.repy]** - * [wiki:SeattleLib/time_interface.repy time_interface.repy] + ## [Time](Time.md) (click for a description) + * [ntp_time.repy](ntp_time.repy.md) + * [tcp_time.repy](tcp_time.repy.md) + * **[time.repy](time.repy.md)** + * [time_interface.repy](time_interface.repy.md) - * [wiki:SeattleLib/DataEncoding Data encoding] (click for a description) - * [wiki:SeattleLib/base64.repy base64.repy] - * [wiki:SeattleLib/binascii.repy binascii.repy] - * **[wiki:SeattleLib/deserialize.repy deserialize.repy]** - * **[wiki:SeattleLib/serialize.repy serialize.repy]** + ## [Data encoding](DataEncoding.md) (click for a description) + * [base64.repy](base64.repy.md) + * [binascii.repy](binascii.repy.md) + * **[wiki::deserialize.repy]** + * **[wserialize.repy](serialize.repy.md)** - * [wiki:SeattleLib/UrlParsingAndXml URL parsing / XML] (click for a description) - * [wiki:SeattleLib/httpretrieve.repy httpretrieve.repy] - * [wiki:SeattleLib/xmlparse.repy xmlparse.repy] - * [wiki:SeattleLib/xmlrpc_client.repy xmlrpc_client.repy] - * [wiki:SeattleLib/xmlrpc_common.repy xmlrpc_common.repy] - * **[wiki:SeattleLib/xmlrpc_server.repy xmlrpc_server.repy]** - * [wiki:SeattleLib/urllib.repy urllib.repy] + ## [URL parsing / XML](UrlParsingAndXml.md) (click for a description) + * [httpretrieve.repy](httpretrieve.repy.md) + * [xmlparse.repy](xmlparse.repy.md) + * [xmlrpc_client.repy](xmlrpc_client.repy.md) + * [xmlrpc_common.repy](xmlrpc_common.repy.md) + * **[xmlrpc_server.repy](xmlrpc_server.repy.md)** + * [urllib.repy](urllib.repy.md) - * [wiki:SeattleLib/ConcurrencyAndParallelism Concurrency / Parallelism] (click for a description) - * [wiki:SeattleLib/cv.repy cv.repy] - * [wiki:SeattleLib/parallelize.repy parallelize.repy] - * [wiki:SeattleLib/semaphore.repy semaphore.repy] - * **[wiki:SeattleLib/uniqueid.repy uniqueid.repy]** + ## [Concurrency / Parallelism](ConcurrencyAndParallelism.md) (click for a description) + * [cv.repy](cv.repy.md) + * [parallelize.repy](parallelize.repy.md) + * [semaphore.repy](semaphore.repy.md) + * **[uniqueid.repy](uniqueid.repy.md)** - * [wiki:SeattleLib/DataRetrieval Data retrieval] (click for a description) - * [wiki:SeattleLib/domainnameinfo.repy domainnameinfo.repy] - * [wiki:SeattleLib/geoip_client.repy geoip_client.repy] - * [wiki:SeattleLib/httpserver.repy httpserver.repy] - * [wiki:SeattleLib/servicelookup.repy servicelookup.repy] + ## [Data retrieval](DataRetrieval.md) (click for a description) + * [domainnameinfo.repy](domainnameinfo.repy.md) + * [geoip_client.repy](geoip_client.repy.md) + * [httpserver.repy](httpserver.repy.md) + * [servicelookup.repy](servicelookup.repy.md) - * [wiki:SeattleLib/NetworkCommunication Network communication] (click for a description) - * [wiki:SeattleLib/getvesselsresources.repy getvesselsresources.repy] + ## [Network communication](NetworkCommunication.md) (click for a description) + * [getvesselsresources.repy](getvesselsresources.repy.md) * [wiki:SeattleLib/Multiplexer.repy Multiplexer.repy] - * [wiki:SeattleLib/nmclient.repy nmclient.repy] - * **[wiki:SeattleLib/NATLayer_rpc.repy NATLayer_rpc.repy]** - * **[wiki:SeattleLib/sockettimeout.repy sockettimeout.repy]** - * **[wiki:SeattleLib/session.repy session.repy]** + * [nmclient.repy](nmclient.repy.md) + * **[NATLayer_rpc.repy](NATLayer_rpc.repy.md)** + * **[sockettimeout.repy](sockettimeout.repy.md)** + * **[session.repy](session.repy.md)** - * [wiki:SeattleLib/ProgrammerResources Programmer resources] (click for a description) - * [wiki:SeattleLib/argparse.repy argparse.repy] - * [wiki:SeattleLib/dylink.repy dylink.repy] - * [wiki:SeattleLib/listops.repy listops.repy] - * [wiki:SeattleLib/math.repy math.repy] - * [wiki:SeattleLib/priority_queue.repy priority_queue.repy] - * **[wiki:SeattleLib/repyunit.repy repyunit.repy]** - * [wiki:SeattleLib/random.repy random.repy] - * [wiki:SeattleLib/repypp.py repypp.py] - * [wiki:SeattleLib/safe_eval.repy safe_eval.repy] - * [wiki:SeattleLib/strace.py strace.py] - * [wiki:SeattleLib/textops.py textops.py] - * [wiki:SeattleLib/urlparse.repy urlparse.repy] - * [wiki:SeattleLib/dnscommon.repy dnscommon.repy] - * [wiki:SeattleLib/bundle.repy bundle.repy] \ No newline at end of file + ## [Programmer resources](ProgrammerResources.md) (click for a description) + * [argparse.repy](argparse.repy.md) + * [dylink.repy](dylink.repy.md) + * [listops.repy](listops.repy.md) + * [math.repy](math.repy.md) + * [priority_queue.repy](priority_queue.repy.md) + * **[repyunit.repy](repyunit.repy.md)** + * [random.repy](random.repy.md) + * [repypp.py](repypp.py.md) + * [safe_eval.repy](safe_eval.repy.md) + * [strace.py](strace.py.md) + * [textops.py](textops.py.md) + * [urlparse.repy](urlparse.repy.md) + * [dnscommon.repy](dnscommon.repy.md) + * [bundle.repy](bundle.repy.md) From 542aeb4c3bcd4498c0db7968cfb3a31957caab50 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 00:36:25 -0400 Subject: [PATCH 061/122] Update README.md --- Contributing/README.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/Contributing/README.md b/Contributing/README.md index 79453454..3e555c4e 100644 --- a/Contributing/README.md +++ b/Contributing/README.md @@ -10,7 +10,7 @@ the following: 1. Start by reading the [Seattle docs](../README.md). This will give you a better sense of what Seattle is and how its structured, and contains pointers - to the programming documentation as well. The [Contributor's Page](ContributorsPage.wiki) + to the programming documentation as well. The [Contributor's Page](ContributorsPage.md) has even more reading material and links. 2. Join our [mailing list](https://groups.google.com/forum/?hl=en#!forum/seattle-devel)! @@ -24,7 +24,7 @@ has even more reading material and links. insights that you may have, and also post any concerns or questions that arise. 4. When you are done with your issue, make sure your code adheres to our - [code style guidelines](../Programming/CodingStyle.wiki). Have someone else on + [code style guidelines](WebCodingStyle.md). Have someone else on the team review your code. When they give you the OK, send a pull request. If your code is good, someone will merge in your changes. From ec0177019cfae43b05578b51ab3a6b900a20a71c Mon Sep 17 00:00:00 2001 From: Aleks Borovtsov Date: Mon, 14 Oct 2019 00:58:18 -0400 Subject: [PATCH 062/122] Update httpserver.repy.md added links to readme --- Programming/SeattleLib_v1/httpserver.repy.md | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/Programming/SeattleLib_v1/httpserver.repy.md b/Programming/SeattleLib_v1/httpserver.repy.md index 4c2961aa..fbd98b13 100644 --- a/Programming/SeattleLib_v1/httpserver.repy.md +++ b/Programming/SeattleLib_v1/httpserver.repy.md @@ -1,18 +1,18 @@ -# httpserver.repy +# httpserver.repy This library abstracts away the details of the HTTP protocol, providing an alternative to calling a user-supplied function on each request. The return value of the user-supplied function determines the response that is sent to the HTTP client. This allows the end user to not have to worry about the semantics of communicating with an HTTP server. In many ways, this module is analogous to the http.server module in Python documentation. See http://docs.python.org/py3k/library/http.server.html. -### Classes & Functions +### Classes & Functions ``` def httpserver_registercallback(addresstuple, cbfunc): ``` Registers a callback function on the (host, port). - Notes: + Notes: * addresstuple is an address 2-tuple to bind to: ('host', port). * cbfunc is the callback function to process requests. It takes one argument, which is a dictionary describing the HTTP request. It looks like this (just an example): @@ -49,21 +49,21 @@ def httpserver_stopcallback(callbackid): ``` Removes an existing callback function, i.e. stopping the server. - Notes: + Notes: * callbackid is the id returned by httpserver_registercallback(). * Raises IndexError or KeyError if the id is invalid or has already been deleted. -### Includes +### Includes -[wiki:SeattleLib/urllib.repy] +[wiki:SeattleLib/urllib.repy](urllib.repy.md) -[wiki:SeattleLib/urlparse.repy] +[wiki:SeattleLib/urlparse.repy](urlparse.repy.md) -[wiki:SeattleLib/uniqueid.repy] +[wiki:SeattleLib/uniqueid.repy](uniqueid.repy.md) -[wiki:SeattleLib/sockettimeout.repy] +[wiki:SeattleLib/sockettimeout.repy](sockettimeout.repy.md) -[wiki:SeattleLib/httpretrieve.repy] +[wiki:SeattleLib/httpretrieve.repy](httpretrieve.repy.md) From 23866683ac6d3157ccb44b368c4351c6cc4c12eb Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 02:01:35 -0400 Subject: [PATCH 063/122] Update ContributorsPage.md ln 12 cannot find the RepoAccess file ln 35,83, 88 is in outdated if you fix these pages you have to fix these links I think some of the other stuff is from the other Seattle pages --- Contributing/ContributorsPage.md | 52 +++++++++++++++----------------- 1 file changed, 25 insertions(+), 27 deletions(-) diff --git a/Contributing/ContributorsPage.md b/Contributing/ContributorsPage.md index b667f9bf..2dd5c596 100644 --- a/Contributing/ContributorsPage.md +++ b/Contributing/ContributorsPage.md @@ -1,37 +1,38 @@ -# Contributor Page +# Contributor Page This page contains information useful to developers working on Seattle. Anything that is essential for each developer to read is **in bold text**. Note: We've recently moved our VCS and ticket system to GitHub. Please see our [organization's page](https://github.com/SeattleTestbed) for the latest code, to discuss open issues, and to create forks off our code base for your contributions! -**If you are a new contributor, please look at our [wiki:GettingStarted Getting Started] page.** +**If you are a new contributor, please look at our [Getting Started](README.md) page.** * Accessing our GitHub repositories, building components, Github & Git tutorial, and using the wiki * Check out (i.e., ```git clone```) our source code from [https://github.com/SeattleTestbed] - * [wiki:BuildInstructions Instructions] for building Seattle components + * [Instructions](BuildInstructions.md) for building Seattle components * [wiki:Local/RepoAccess Github and Git] instructions * [Useful Git Commands](http://zackperdue.com/tutorials/super-useful-need-to-know-git-commands) * [How to resolve a merge conflict with Git](https://help.github.com/articles/resolving-a-merge-conflict-from-the-command-line) - * [wiki:SubmittingAPatch How to submit a patch for inclusion if you do not have commit access] - * ~~**[wiki:ContributorAccounts Getting accounts] for Seattle Wiki, testbed machines, and seattle-devel**~~ - * ~~**[wiki:ManagingTracTickets Managing Trac tickets]**~~ + * [How to submit a patch for inclusion if you do not have commit access](SubmittingAPatch.md) * Programming in Repy * The [Python Tutorial](http://docs.python.org/tutorial/) from the Python site. - * [wiki:PythonTutorial Subset of Python Tutorial] with basic information about the subset of Python needed to write code in Repy - * [wiki:PythonVsRepy Python You Need to Forget to Use Repy] - * **[wiki:RepyTutorial Repy Tutorial]** - * [wiki:RepyApi Repy Library] reference - * [wiki:SeattleLib Seattle Standard Library] reference - * [wiki:PortingPythonToRepy Porting Python to Repy Guide] - * [wiki:RepyHelper Importing Repy code into Python using repyhelper] - * [wiki:RepyV1vsRepyV2 Difference between Repy V1 and Repy V2] + * [PythonTutorial Subset of Python Tutorial](../Programming/PythonTutorial.md) with basic information about the subset of Python needed to write code in Repy + * [Python Vs Repy Python You Need to Forget to Use Repy](../Programming/PythonVsRepy.md) + * [Python vs RePyV2 Python You need to Forget to Use RePyV2](../Programming/PythonVsRepyV2.md) + * **[Repy Tutorial](../Programming/RepyTutorial.md)** + * **[RePyV2 Tutorial](../Programming/RepyV2Tutorial.md)** + * [Repy Library](../Programming/RepyApi.md) reference + * [RepyV2 Library](../Programming/RepyV2Api.md) reference + * [Seattle Standard Library](../Programming/SeattleLib_v1/SeattleLib.md) reference + * [Porting Python to Repy Guide](../Programming/PortingPythonToRepy.md) + * [Importing Repy code into Python using repyhelper](../Programming/RepyHelper.md) + * [Difference between Repy V1 and Repy V2](../Programming/RepyV1vsRepyV2.md) * Testing - * **[wiki:UnitTestFramework Writing Tests for Seattle's Unit Test Framework]** + * **[Writing Tests for Seattle's Unit Test Framework](UnitTestFramework.md)** * **[wiki:UnitTests Running Repy and / or node manager unit tests]** - * [wiki:UnitTestFrameworkRunning Supplemental information about Running Tests with Seattle's Unit Test Framework] - * [wiki:UpdaterUnitTests Running Software Updater Unit Tests] + * [Supplemental information about Running Tests with Seattle's Unit Test Framework](UnitTestFrameworkRunning.md) + * [wiki:UpdaterUnitTests Running Software Updater Unit Tests](../Outdated/UpdaterUnitTests.md) * [wiki:IntegrationTestFramework Writing Integration Tests] * [wiki:Local/RunningIntegrationTests Running Integration Tests] * [wiki:ClearinghouseInstallation Running a Clearinghouse test server] @@ -43,7 +44,7 @@ Note: We've recently moved our VCS and ticket system to GitHub. Please see our [ * Style guidelines for documentation, python code, web code * **[Python/Repy style guidelines](https://github.com/secure-systems-lab/code-style-guidelines) for most Seattle programming** - * [wiki:WebCodingStyle Web style guidelines] for web programming + * [Web style guidelines](WebCodingStyle.md) for web programming * **[wiki:Local/WikiFormatting Documentation format guidelines] for wiki pages** * Installation instructions and documentation @@ -70,9 +71,7 @@ Note: We've recently moved our VCS and ticket system to GitHub. Please see our [ * [wiki:Libraries/ExperimentLibrary Experiment Library] for use in scripting communication with nodes, etc. * [wiki:DevelopingWithEclipse Developing with Eclipse] - * ~~Project Resources~~ - * ~~[wiki:SeattleResources A full listing of Seattle project services]~~ - * ~~[wiki:Local/ContributorAccountManagement Creating and deleting accounts for developers]~~ + * Project Resources * [wiki:SeattleAdminResources Resources needed by Seattle sysadmin] * Events relevant to Seattle and Seattle team information @@ -81,11 +80,10 @@ Note: We've recently moved our VCS and ticket system to GitHub. Please see our [ * Other * [wiki:SeattleBackend Seattle Backend] - * [wiki:ContainmentInSeattle] about controlling node outgoing traffic - * ~~[wiki:Archive/MobileCeNotes Windows CE] implementation notes and problems~~ + * [Containment In Seattle](../Outdated/ContainmentInSeattle.md) about controlling node outgoing traffic * [wiki:BenchmarkCustomInstallerInfo Benchmark and Custom Installer Info] implementation notes and problems. - * [wiki:RepyNetworkRestrictions Repy Network Restrictions] information and errata. + * [Repy Network Restrictions](../Programming/RepyNetworkRestrictions.md) information and errata. * [wiki:Libraries/StatisticsLibrary Statistics Library] for analyzing date regarding nodes & VMs on Seattle. - * [wiki:Applications/GeoIpServer Running a GeoIP Server] - * How to run the [wiki:RunningSecLayerBenchmarks security layer benchmarks] - * [SeleXor](http://selexor.poly.edu/) for additional control over the VM acquisition process. \ No newline at end of file + * [Running a GeoIP Server](../Applications/GeoIpServer.md) + * How to run the [Security Layer benchmarks](../OutdatedRunningSecLayerBenchmarks.md) + * [SeleXor](http://selexor.poly.edu/) for additional control over the VM acquisition process. From 3c111c1f17af0c17de3126a73b7b53676cb68a1c Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 02:02:50 -0400 Subject: [PATCH 064/122] Update LipstickOnAPigExceptionHierarchy.md --- Archive/LipstickOnAPigExceptionHierarchy.md | 60 ++++++++++----------- 1 file changed, 30 insertions(+), 30 deletions(-) diff --git a/Archive/LipstickOnAPigExceptionHierarchy.md b/Archive/LipstickOnAPigExceptionHierarchy.md index add559f1..0603f407 100644 --- a/Archive/LipstickOnAPigExceptionHierarchy.md +++ b/Archive/LipstickOnAPigExceptionHierarchy.md @@ -1,9 +1,9 @@ -# Lipstick on a Pig Exception Hierarchy Proposal +# Lipstick on a Pig Exception Hierarchy Proposal -## Overview / Purpose +## Overview / Purpose -The [wiki:Archive/LipstickOnAPig Lipstick on a Pig] strike force is attempting to create a standard exception hierarchy for repy to provide a unified method to allow users to handle exceptions within repy programs. +The [Lipstick on a Pig](LipstickOnAPig.md) strike force is attempting to create a standard exception hierarchy for repy to provide a unified method to allow users to handle exceptions within repy programs. Currently the Repy library has inconsistent treatment and use of exceptions, which can be confusing and off-putting for new users. Our goal is to come up with a clear, uniform and logical way to present exceptions. The repy library functions have specific needs and goals, and the reflection hierarchy we're proposing reflects this. We are focusing on creating the core exception hierarchy which will be used by the repy library calls themselves; exceptions specific to any seattlelib code will need to augment the hierarchy with their own exceptions. @@ -15,9 +15,9 @@ The hierarchy we designed is meant to account for the specific types of errors w ---- - -## Proposed Repy Hierarchy + +## Proposed Repy Hierarchy ---- Below is our proposed exception hierarchy. The RepyException tree is separate from Exception to show how the two are meant to be distinct, although RepyException is a subclass of Python's Exception. @@ -53,21 +53,21 @@ PythonException +-- Builtin exceptions raised due to programmer mistakes (AttributeError, IndexError, NameError, etc) ``` - -## Comparison to Python and Current Repy Implementation + +## Comparison to Python and Current Repy Implementation ---- The python builtins tend to throw the same types of errors, and repy is meant to mimic this behavior. For example, builtins overwhelmingly raise ```TypeError``` when passed an argument of the wrong type, and raise ```ValueError``` when passed an incorrect value (such as a positive number instead of a negative one). Repy exceptions are also meant to be less cryptic than some of the python exceptions. The socket.* errors for example were quite cryptic for new users, which resulted in the creation of ```AddressError``` rather than using python's ```socket.gaierror```. - -### Reused Python Exceptions + +### Reused Python Exceptions ---- -#### TypeError +#### TypeError The python library uses [TypeError](http://docs.python.org/library/exceptions.html#exceptions.TypeError) to notify users that an argument of the wrong type was passed to a function. We want to emulate this behavior, as the current repy implementation primarily raises ```Exception``` and gives the description of what argument was incorrect. For example, the python ```abs``` built-in expects a numeric type. If a string is passed in, it raises a ```TypeError```. @@ -76,7 +76,7 @@ Similar behaviour is proposed for repy. For example, if a user tries specifying -#### ValueError +#### ValueError We want to change the Repy Library to emulate python's builtin API's usage of [ValueError](http://docs.python.org/library/exceptions.html#exceptions.ValueError). Although exact Python behavior differs per implementation and per call (a bad thing, in our opinion) ```ValueError``` tends to be used to indicate that a value of the correct type was passed in, but an invalid value was used. An example of this is the ```sleep``` API call, which expects a numeric value greater than 0. If a negative number is passed in, a ValueError should be raised @@ -105,9 +105,9 @@ socket.gethostbyname_ex("") ``` While this could be considered a network error, it seems reasonable to assume that getting host information from the empty string is obviously a value error, no network checks have to be performed. The repy version of the above code should raise a ```ValueError``` - -### Network Examples + +### Network Examples ---- Network failures can be of particular importance given the nature of repy and the types of programs that tend to get written in it. There are a large number of reasons for why network operations can fail, and we feel python's socket module's interface could be improved upon. @@ -116,7 +116,7 @@ Native python's exception types for network related errors are pretty limited. T Our goal is to make it easy for users to determine precisely what went wrong with a network call, and be able to recover gracefully from these error conditions. -#### Host Name Example +#### Host Name Example In native python, socket errors are defined in the socket module. Since repy has no concept of modules, and doesn't allow imports, it doesn't make sense to use the code snippet below. ```python import socket @@ -137,7 +137,7 @@ except NetworkAddressError: #recover from error ``` -#### NetworkAddressError +#### NetworkAddressError One class of errors that users may tend to see often due to the nature of resolution is address errors. In Python these are presented as ```socket.gaierror```, and what the specific cause of the problem is presented in the text of the exception. Currently several repy network API calls still make use of ```socket.*``` errors. However the socket module is not in the scope of repy user programs, and modules aren't allowed to be imported. This means the current implementation of repy requires workarounds: @@ -167,7 +167,7 @@ except NetworkError: #handle error condition ``` -#### Connection Example +#### Connection Example The way of interacting with sockets is also different from regular python: ```python @@ -200,7 +200,7 @@ finally: The exception handing for both cases are meant to be similar. We aimed to keep the programming model familiar enough to programmers with a python background, but also more user friendly to those just staring out with repy. Just as ```openconn``` and ```sendmess``` provide easy access for users to interact over networks, ```NetworkError``` and its subclasses are meant to facilitate convenient handling of common errors that can occur. -#### ConnectionRefusedError +#### ConnectionRefusedError In python, the following code snippet raises a ```socket.error```: ```python import socket @@ -217,7 +217,7 @@ s = openconn('127.0.0.1', 12347, '127.0.0.1', 12345) #a ConnectionRefusedError would be raised ``` -#### NetworkDisconnectedError +#### NetworkDisconnectedError This error is meant to be raised by ```getmyip()```, to indicate that a computer lost it's connection to the internet. The current behavior is ambiguous; ```getmyip``` raises a ```socket.gairerror``` with the message No address associated with hostname". This error should also be raised if the socket layer raises ```socket.error``` 101: "Network is unreachable". This is currently the exception that is thrown by various network API calls, such as when ```sendmess``` is given an external ip but the network is disconnected @@ -225,16 +225,16 @@ This error should also be raised if the socket layer raises ```socket.error``` 1 - -### File System Examples + +### File System Examples ---- While the python standard library provides many functions to flexibly interface with the underlying file system, the nature of repy requires a much more restricted subset of allowed operations. Additionally, there are a number of areas (such as file system modes) where the python standard library functions fail "silently" by either not raising an exception or proceeding with unexpected behavior. The goal of the repy file system exceptions are to make it very clear what error has occurred and to not silently continue. -#### File Open Examples +#### File Open Examples In native python, most errors with the file system raise a general "IOError" with more specific details in the message. This makes it hard to realize what happened: @@ -257,7 +257,7 @@ except FileNotFoundError: pass ``` -#### File Open Mode Example +#### File Open Mode Example Python "silently" allows some obscure modes with the file system: @@ -278,7 +278,7 @@ except InvalidFileModeException: pass ``` -#### Illegal File Names +#### Illegal File Names Python allows many characters within the open method such as ".." to traverse directories. Because Repy has no concept of "directories" for programs running in the VMs, it must be more restrictive of allowed file names. @@ -299,9 +299,9 @@ except IllegalFileNameException: pass ``` - -### Resource Errors + +### Resource Errors ---- Given the nature of repy, it is necessary to be able to signal users that they have exceeded their allowed resources. There are several types of resources, and some of these errors may be recoverable, some may not. For example, when a program is consuming too much memory, there is no way for a program to catch that exception. However, if a user tries to have too many files open or have too many event handlers at once, it will be possible to catch the exception raised during the attempt to create more, and allow the programmer to take appropriate action. @@ -310,9 +310,9 @@ Recoverable resource errors generally fall into two categories: attempting to us When trying to use a resource that wasn't permitted in a VMl's restrictions file, a ResourceNotAllowedError will be raised. In the case of overusing a resource, a ResourceExhaustedError will be raised. These will make it more clear why an error occured when trying to use a node's resources, and avoid the hacky string parsing currently required for recovery. - -## Internal Errors + +## Internal Errors ---- The above errors are meant to be the exceptions exposed to users. If a corrupt internal state is detected by repy, an ```InternalError``` will be raised. When this occurs, a user's repy program will terminate instantly. The rationale behind this is that we want to hide potential repy weaknesses to typical users, and ensure that a corrupt VM doesn't resume execution in a potentially harmful state. @@ -324,12 +324,12 @@ If this occurs in a ```try...finally``` block, the finally block will not get ex Information regarding the internal error will get privately logged, to provide seattle developers some insight regarding what caused the problem, and do postmortem analysis. - -## Unresolved Questions/Issues + +## Unresolved Questions/Issues ---- There are a number of problems/questions we weren't sure the most appropriate way to handle. We'd appreciate any feedback regarding these issues * What type of error should be raised if a user's IP address changes? If a program has event handlers associated with an IP address and that changes, how should the user be notified? * The biggest concern for changing IP's is how to notify the user. It's easy to imagine the scenario where the only thing keeping a program in execution (rather than terminating) is event handlers waiting to be triggered. So if a program has a recvmess handler waiting to receive UDP packets, but the machine's IP changes, how do we notify the user in a meaningful way? If there's no context to which allow them to catch an exception we raised, does it even make sense to? Plus, recvmess is provided a localip, which is no longer valid! It's conceivable that users would write their own logic to periodically check the IP, and if it changes, then to update the event handlers accordingly. As to how/if repy should detect these (and possibly unregister stale listeners) is still uncertain... - * Should an exception should be thrown if the interface and destination IP don't match? I.e. ```sendmess("google.com",80, "hi", "127.0.0.1", 12345)``` \ No newline at end of file + * Should an exception should be thrown if the interface and destination IP don't match? I.e. ```sendmess("google.com",80, "hi", "127.0.0.1", 12345)``` From caaf858ce7643d8cde3bc84af6d803d1ae12d167 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 02:22:54 -0400 Subject: [PATCH 065/122] Update SeattleShellBackend.md --- UnderstandingSeattle/SeattleShellBackend.md | 32 ++++++++++----------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/UnderstandingSeattle/SeattleShellBackend.md b/UnderstandingSeattle/SeattleShellBackend.md index c1aab34a..6ce93147 100644 --- a/UnderstandingSeattle/SeattleShellBackend.md +++ b/UnderstandingSeattle/SeattleShellBackend.md @@ -1,10 +1,10 @@ -Seash (also known as Seattle Shell) is the command line interface for interacting with Seattle VMs. This will describe how seash is designed. If you are a user interested in learning how to use Seash, please go to the [SeattleShell main Seattle Shell page]. +Seash (also known as Seattle Shell) is the command line interface for interacting with Seattle VMs. This will describe how seash is designed. If you are a user interested in learning how to use Seash, please go to the [SeattleShell main Seattle Shell page](SeattleShell.md). - -## Command Dictionary + +## Command Dictionary ---- The command dictionary contains command nodes that determine how seash responds to various input. Each command node represents a recognized command keyword, and may have child command nodes that represent keywords that should come after that command. Seash will use this dictionary to check if an input string matches a path through the command dictionary, and if it does, executes the command at the terminating node. @@ -48,16 +48,16 @@ command_dictionary = { ... } ``` - -## Defining New Modules + +## Defining New Modules ---- Seash has support for adding additional commands via modules that are imported on startup. These modules should be placed in a subdirectory within the /modules/ folder. These modules are standard python packages. - -### Creating a Module + +### Creating a Module Defining a module is relatively straightforward. First, create the ```__init__.py``` file that contains the ```moduledata``` dictionary in the module namespace that is loaded by the module importer. It should look like the following: ```python @@ -70,9 +70,9 @@ moduledata = { 'input_preprocessor': input_preprocessor, } ``` - -#### The Command Dictionary + +#### The Command Dictionary The command_dict defines all the commands that are to be part of the module. These command_dicts should be specified in a similar manner as described in the command dictionary section above. @@ -90,9 +90,9 @@ command_dict = { 'summary':'Display the latitude & longitude of the node', 'help_text':"""...""",} ``` - -#### Module Level Documentation + +#### Module Level Documentation You must also have module-level documentation. ```python @@ -105,9 +105,9 @@ Clearinghouse, use the 'browse' command, and then in any group, run either 'show or 'show coordinates'. """ ``` - -#### Automatic Updating + +#### Automatic Updating You must also specify an update URL to where your module can be found. This will be used with the updater to automatically update the module. To disable this feature, set the URL to ```None```. Note, this is not functional yet. All modules should not have this set until then. ```python # This is already set in the moduledata above so this is no longer needed. @@ -116,9 +116,9 @@ You must also specify an update URL to where your module can be found. This wil moduledata['url'] = "http://update.url/" ``` - -#### Input Preprocessor + +#### Input Preprocessor The input preprocessor is available for modules to tap into raw input from the command line to perform initial preprocessing. This is used especially in the variables module. This preprocessor will be given one string as input, and must return one string representing the results of the preprocessing. This example is a simple example that replaces every instance of $USERPORT with a user's port. @@ -135,4 +135,4 @@ def input_preprocessor(user_input): else: retstr += user_input return retstr -``` \ No newline at end of file +``` From 9785f93e55b2c252a38add1037506f758a6b9949 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 02:24:18 -0400 Subject: [PATCH 066/122] Update ContributorsPage.md Im assuming Seattle Backend is the seattle shell backend --- Contributing/ContributorsPage.md | 40 ++++++++++++++++---------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/Contributing/ContributorsPage.md b/Contributing/ContributorsPage.md index 2dd5c596..3dcc9ba3 100644 --- a/Contributing/ContributorsPage.md +++ b/Contributing/ContributorsPage.md @@ -9,7 +9,7 @@ Note: We've recently moved our VCS and ticket system to GitHub. Please see our [ * Check out (i.e., ```git clone```) our source code from [https://github.com/SeattleTestbed] * [Instructions](BuildInstructions.md) for building Seattle components - * [wiki:Local/RepoAccess Github and Git] instructions + * [Github and Git](../Archive/Local/RepoAccess.md) instructions * [Useful Git Commands](http://zackperdue.com/tutorials/super-useful-need-to-know-git-commands) * [How to resolve a merge conflict with Git](https://help.github.com/articles/resolving-a-merge-conflict-from-the-command-line) * [How to submit a patch for inclusion if you do not have commit access](SubmittingAPatch.md) @@ -32,11 +32,11 @@ Note: We've recently moved our VCS and ticket system to GitHub. Please see our [ * **[Writing Tests for Seattle's Unit Test Framework](UnitTestFramework.md)** * **[wiki:UnitTests Running Repy and / or node manager unit tests]** * [Supplemental information about Running Tests with Seattle's Unit Test Framework](UnitTestFrameworkRunning.md) - * [wiki:UpdaterUnitTests Running Software Updater Unit Tests](../Outdated/UpdaterUnitTests.md) - * [wiki:IntegrationTestFramework Writing Integration Tests] - * [wiki:Local/RunningIntegrationTests Running Integration Tests] - * [wiki:ClearinghouseInstallation Running a Clearinghouse test server] - * [wiki:Archive/CustomInstallerBuilderTesting Running a Custom Installer Builder test server] + * [Running Software Updater Unit Tests](../Outdated/UpdaterUnitTests.md) + * [Writing Integration Tests](../Operating/IntegrationTestFramework.md) + * [Running Integration Tests](../Archive/Local/RunningIntegrationTests.md) + * [Running a Clearinghouse test server](../Operating/Clearinghouse/Installation.md) + * [Running a Custom Installer Builder test server](../Archive/CustominstallerBuilderTesting.md) * [wiki:RemoteTestingService Running remote tests on seattle testbeds] * **Continuous Integration via Travis-CI and AppVeyor:** * [For team members - managing CI](https://github.com/SeattleTestbed/seattlelib_v2/wiki/Continuous-Integration-for-the-Team) @@ -45,10 +45,10 @@ Note: We've recently moved our VCS and ticket system to GitHub. Please see our [ * Style guidelines for documentation, python code, web code * **[Python/Repy style guidelines](https://github.com/secure-systems-lab/code-style-guidelines) for most Seattle programming** * [Web style guidelines](WebCodingStyle.md) for web programming - * **[wiki:Local/WikiFormatting Documentation format guidelines] for wiki pages** + * **[Documentation format guidelines](../Archive/Local/WikiFormatting.md) for wiki pages** * Installation instructions and documentation - * [wiki:InstallerDocumentation Formal Installer Documentation] + * [Formal Installer Documentation](../UnderstandingSeattle/InstallerDocumentation.md) * [wiki:SeattleDownload Download and installation instructions] for Seattle * [wiki:ClearinghouseInstallation Installing your own instance of Seattle Clearinghouse] * [wiki:CustomInstallerBuilder Using the Custom Installer Builder] @@ -59,31 +59,31 @@ Note: We've recently moved our VCS and ticket system to GitHub. Please see our [ * [wiki:CustomInstallerBuilderApi Custom Installer Builder XML-RPC API] * Deployment - * [wiki:BaseInstallers Building] the base installers + * [Building](../Operating/BaseInstallers.md) the base installers * [wiki:NsisSystemSetup System setup instructions] to be able to build the Windows GUI installer - * [wiki:Local/VersionDeployment Deploying] a new version - * [wiki:SoftwareUpdaterSetup Setting up] the software updater - * [wiki:Archive/SeattleGeniProductionHttp Updating] the production Seattle Clearinghouse code - * [wiki:BuildDemokit Building the Demokit] - * [wiki:Libraries/Overlord Overlord Deployment and Monitoring Library] for deploying persistent services on VMs + * [Deploying](../Archive/Local/VersionDeployment.md.md) a new version + * [Setting up](../Operating/SoftwareUpdaterSetup.md) the software updater + * [Updating](../Archive/SeattleGeniProductionHttp.md) the production Seattle Clearinghouse code + * [Building the Demokit](../Operating/BuildDemokit.md) + * [Overlord Deployment and Monitoring Library](../Archive/Libraries/Overlord.md) for deploying persistent services on VMs * Developer Resources - * [wiki:Libraries/ExperimentLibrary Experiment Library] for use in scripting communication with nodes, etc. + * [Experiment Library](../Archive/Libraries/ExperimentLibrary.md) for use in scripting communication with nodes, etc. * [wiki:DevelopingWithEclipse Developing with Eclipse] * Project Resources * [wiki:SeattleAdminResources Resources needed by Seattle sysadmin] * Events relevant to Seattle and Seattle team information - * [wiki:SeattleTalks Upcoming talks] about Seattle - * **[wiki:Local/ContributorContactInfo Project Members] contact information** + * [Upcoming talks](../SeattleTalks.md) about Seattle + * **[Project Members](../Archive/Local/ContributorContactInfo.md) contact information** * Other - * [wiki:SeattleBackend Seattle Backend] + * [Seattle Backend](../UnderstandingSeattle/SeattleShellBackend.md) * [Containment In Seattle](../Outdated/ContainmentInSeattle.md) about controlling node outgoing traffic - * [wiki:BenchmarkCustomInstallerInfo Benchmark and Custom Installer Info] implementation notes and problems. + * [Benchmark and Custom Installer Info](../UnderstandingSeattle/BenchmarkCustomInstallerInfo.md) implementation notes and problems. * [Repy Network Restrictions](../Programming/RepyNetworkRestrictions.md) information and errata. - * [wiki:Libraries/StatisticsLibrary Statistics Library] for analyzing date regarding nodes & VMs on Seattle. + * [Statistics Library](../Archive/Libraries/StatisticsLibrary.md) for analyzing date regarding nodes & VMs on Seattle. * [Running a GeoIP Server](../Applications/GeoIpServer.md) * How to run the [Security Layer benchmarks](../OutdatedRunningSecLayerBenchmarks.md) * [SeleXor](http://selexor.poly.edu/) for additional control over the VM acquisition process. From 2a878aaadddf7d0f8d73fada4fd813c14ecb0ebc Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 02:35:59 -0400 Subject: [PATCH 067/122] Update SecurityLayerPartOne.md --- .../SecurityLayerPartOne.md | 61 +++++++++---------- 1 file changed, 30 insertions(+), 31 deletions(-) diff --git a/EducationalAssignments/SecurityLayerPartOne.md b/EducationalAssignments/SecurityLayerPartOne.md index f5707827..b1814ad6 100644 --- a/EducationalAssignments/SecurityLayerPartOne.md +++ b/EducationalAssignments/SecurityLayerPartOne.md @@ -1,5 +1,5 @@ -# Building a Security Layer +# Building a Security Layer This assignment will help you understand security mechanisms. You will be guided through the steps of creating a reference monitor using the security layer functionality in Repy V2. A reference monitor is an access control concept that refers to an abstract machine @@ -12,9 +12,9 @@ This assignment is intended to prepare you for thinking about security paradigms - -## Overview + +## Overview ---- In this assignment an attacker attempts to write "MZ" to the first two characters of a file. (Historically, this is an indicator for an [old executable format](http://www.fileformat.info/format/exe/corion-mz.htm).) Your goal is to prevent the attacker from succeeding. Thus you must stop a program from creating a file with "MZ" as the first two characters. You will do this by adding security rules to the functions available for reading from a file and writing to a file. The future of the system depends on your ability to write secure code! @@ -25,9 +25,9 @@ Three design paradigms are at work in this assignment: accuracy, efficiency, and * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. This means you should not do things like re-read the file before each write. * Security: The attacker should not be able to circumvent the security layer. Hence, if "MZ" can be written two the first two characters, the security is compromised. - -### Getting Python + +### Getting Python ---- Please note you must have Python 2.6 or 2.7 to complete this assignment. Instructions on how to get Python for Windows can be found [InstallPythonOnWindows here]. If you are using Linux or a Mac it is likely you already have Python. In order to verify this, simply open a terminal and type ```python```. Please check its version on the initial prompt. @@ -35,7 +35,7 @@ Please note you must have Python 2.6 or 2.7 to complete this assignment. Instruc -### Getting RepyV2 +### Getting RepyV2 ---- The preferred way to get Repy is ''installing from source''. For this, you check out the required Git repositories, and run a build script. ([wiki:BuildInstructions This page] describes the process in greater detail). You can always update the repos later, and rebuild, so that you get the latest stable version of the Repy runtime. @@ -84,7 +84,7 @@ In order to test whether or not these steps worked, please copy and paste the co If you got an error, please go through the troubleshooting section found below. -### Troubleshooting Repy code +### Troubleshooting Repy code If you can't get Repy files to run, some of the following common errors may have occurred: * using `print` instead of `log`: @@ -108,25 +108,25 @@ To run the unit test, which will automatically tell you if you have errors with * [wiki:RepyV2CheckoutAndUnitTests] --> - -### Tutorials for Repy and Python + +### Tutorials for Repy and Python ---- Now that you have Repy and Python, you may need a refresher on how to use them. The following tutorials are excellent sources of information. * Python tutorial: **[http://docs.python.org/tutorial/]** * Seattle tutorial: **[https://seattle.poly.edu/wiki/PythonVsRepy]** * list of RepyV2 syntax: **[wiki:RepyV2API]** - -## Building the security layer + +## Building the security layer ---- **[wiki:RepyV2SecurityLayers]** explains the syntax used to build a reference monitor. The general tutorials above will aid in looking up other details about Repy. Remember, you have no idea how the attacker will try to penetrate your security layer, so it is important that you leave nothing to chance! Your reference monitor should try to stop every attack you can think of. Not just one or two. A good reference monitor will do this and incorporate the design paradigms of accuracy, efficiency and security defined above. - -## A basic (and inadequate) defense + +## A basic (and inadequate) defense Time to start coding! Let's inspect a basic security layer. @@ -199,25 +199,25 @@ CHILD_CONTEXT_DEF["openfile"] = {TYPE:OBJC,ARGS:(str,bool),EXCP:Exception,RETURN secure_dispatch_module() ``` -## Using the example layer +## Using the example layer Keep in mind the above security layer would only stop one kind of attack. Thus if an attacker doesn't know much about file input/output this will probably stop them. However there are a bunch of tricks that can be used in order to circumvent this security layer easily. For instance, the above reference monitor isn't thread safe. For an introduction to thread safety please read [wiki/Thread_safety](http://en.wikipedia.org/wiki/Thread_safety). - -### Code analysis + +### Code analysis The `writeat()` function attempts follow the design principles of accuracy, efficiency, and security. However it does so inadequately. Accuracy is attempted by only blocking "MZ" from being written. If any other characters are written, the `writeat()` function executes normally. Efficiency is preserved because the number of test cases is small. Also security is attempted because "MZ" cannot be written to a file as easily. It is important to note that no security system is perfect. It is more important to make a security system that is infeasible to break, rather than impossible to break. However this security layer does not yet create the necessary infeasibility requirements for most attacks. Thus we see that there is a grey area of what is an acceptable level of impedance. Now let's look at the structure of `writeat()`. First off, the if-statement contains the mitigation. If the "alarm" is not tripped then the `writeat()` function is executed normally. Within the if-statement `data.startswith("MZ")` is used to check for an illegal write. Specifically, `data` is a variable used to store information which will be written to the file. And the method `startswith()` checks to see if the file begins with certain characters, in this case "MZ". The second condition, `(offset == 0)` determines the starting point of the cursor before it writes to the file. Thus offset = 0, means we start writing at the beginning of the file. Similarly if offset =4, the computer would start writing to the 5th position of the file. Starting with 0 rather than 1 has heuristic meaning in computer science. - -## Testing your security layer + +## Testing your security layer ---- In this part of the assignment you will pretend to be an attacker. Remember the attacker's objective is to write "MZ" to a file. By understanding how the attacker thinks, you will be able to write better security layers. Perhaps while attacking your security layer you will think of a new mitigation that should have been implemented. Keep in mind attacks are attempts to mitigate a given security protocol. If even one case succeeds, then your security layer has been compromised. Thus the attack you write should include several methods of attempting to write "MZ" to a file. An example of an attack is found below: @@ -231,13 +231,12 @@ try: # It raised an Exception (as it was supposed to): except ValueError: - log("The security layer correctly blocked the write!!! + log("The security layer correctly blocked the write!!! n") # No Exception was raise else: - log("Wrote an MZ!!! -n") + log("Wrote an MZ!!! \n") finally: # Close the file after our attempt. @@ -246,7 +245,7 @@ finally: **Note:** All attacks should be written as Repy files, using the .r2py extension. -### Code Analysis +### Code Analysis It is important to keep in mind that only lowercase file names are allowed. So in the above code, specifically: ``` @@ -259,9 +258,9 @@ look.txt is a valid file name, however Look.txt is not. Examples of other inval This code attempts to write "MZ" to the file directly. First the file is opened using `myfile=openfile("look.txt",True)`. Next `myfile.writeat("MZ",0)` tries to write "MZ" to the file. The 0 refers to an offset of zero. The `try:` statement tells the program to "try" this case. Notice that the `except` is executed if an error is raised. If the security layer fails the test then the else statement is executed. The `finally:` statement will always run, closing the file. - -### Running your security layer + +### Running your security layer ---- Finally, type the following commands at the terminal to run your security layer with your attack program @@ -269,10 +268,10 @@ Finally, type the following commands at the terminal to run your security layer '''Make sure you use filenames that are all lowercase letters! - -# Notes and Resources + +# Notes and Resources ---- * A list of command line utilities for Windows can be found at **[http://commandwindows.com/command3.htm]** @@ -297,18 +296,18 @@ Finally, type the following commands at the terminal to run your security layer * In repy log replaces print from python. This may be helpful when testing if Repy installed correctly. - -# Extra Credit + +# Extra Credit ---- For extra credit, make a second security layer which stops an attacker from writing 'p0wnd' anywhere in a file. This security layer will be similar to the first one you wrote, except now you must handle the case of `offset != 0` as well as a few other things. You should also write an attack layer which attempts to write 'p0wnd' to a file. **Hint:** Since this string can be written anywhere in the file, there will most likely be more test cases. - -# What to turn in? + +# What to turn in? ---- * Turn in a repy file called reference_monitor_[ name or id number].r2py - * For extra credit turn in a second repy file called extra_credit_[ name or id number].r2py \ No newline at end of file + * For extra credit turn in a second repy file called extra_credit_[ name or id number].r2py From 9d765ae9adea1312ed6de54bdbefed5d06fb38f4 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Mon, 14 Oct 2019 02:37:36 -0400 Subject: [PATCH 068/122] Update EducatorsPage.md --- EducationalAssignments/EducatorsPage.md | 32 ++++++++++++------------- 1 file changed, 16 insertions(+), 16 deletions(-) diff --git a/EducationalAssignments/EducatorsPage.md b/EducationalAssignments/EducatorsPage.md index f9166931..c0bd9147 100644 --- a/EducationalAssignments/EducatorsPage.md +++ b/EducationalAssignments/EducatorsPage.md @@ -1,28 +1,28 @@ -# Educator Portal -This page contains information for educators wishing to use Seattle in the classroom. We would love for you to tell us about your experiences using Seattle in the classroom. For a quick intro you might also want to watch our [wiki:UnderstandingSeattle/DemoVideo five-minute demo] of Seattle in action! If you have any questions about any of the materials on this page, please contact ``` jcappos -AT- poly -DOT- edu ``` +# Educator Portal +This page contains information for educators wishing to use Seattle in the classroom. We would love for you to tell us about your experiences using Seattle in the classroom. For a quick intro you might also want to watch our [five-minute demo](../UnderstandingSeattle/DemoVideo.md) of Seattle in action! If you have any questions about any of the materials on this page, please contact ``` jcappos -AT- poly -DOT- edu ``` -## Networking / Distributed Systems Assignments +## Networking / Distributed Systems Assignments -### Complete Assignments +### Complete Assignments The following set of assignments have complete descriptions, and have fully documented instructor solutions. These assignments are geared toward an undergraduate class in networks or distributed systems. - * The [wiki:EducationalAssignments/TakeHome Take-Home Assignment] introduces [wiki:SeattleShell seash (Seattle Shell)] and teaches about connectivity on the Internet. This is a great starting point to understand how Seattle can help you in a classroom setting. It's also a nice demo for classes that don't have programming assignments, but want to illustrate practical networking concepts. + * The [Take-Home Assignment](TakeHome.md) introduces [Seattle Shell](../UnderstandingSeattle/SeattleShell.md) and teaches about connectivity on the Internet. This is a great starting point to understand how Seattle can help you in a classroom setting. It's also a nice demo for classes that don't have programming assignments, but want to illustrate practical networking concepts. - * The [wiki:EducationalAssignments/StopAndWait Stop and Wait Assignment] has students implement a reliable messaging protocol on top of UDP. This is a good first assignment in that it is simple to code, but gets students familiar with Seattle and allows them to measure practical Internet properties. For comparison, Part 1 of [this assignment](http://www.cs.washington.edu/education/courses/461/06wi/HW/HW2/index.html) is a C version of the assignment. + * The [Stop and Wait Assignment](StopAndWait.md) has students implement a reliable messaging protocol on top of UDP. This is a good first assignment in that it is simple to code, but gets students familiar with Seattle and allows them to measure practical Internet properties. For comparison, Part 1 of [this assignment](http://www.cs.washington.edu/education/courses/461/06wi/HW/HW2/index.html) is a C version of the assignment. - * The [wiki:EducationalAssignments/SlidingWindow Sliding Window Assignment] is an advanced extension to the Stop and Wait Assignment above. + * The [Sliding Window Assignment](SlidingWindow.md) is an advanced extension to the Stop and Wait Assignment above. - * The [wiki:EducationalAssignments/LinkState Link State Routing Assignment] has students implement a link state routing protocol over the Internet. Students implement Dijkstra's shortest path routing and then apply this to route packets in an attempt to minimize latency. + * The [Link State Routing Assignment](LinkState.md) has students implement a link state routing protocol over the Internet. Students implement Dijkstra's shortest path routing and then apply this to route packets in an attempt to minimize latency. - * The [wiki:EducationalAssignments/WebServer Web Server Assignment] (based on [this assignment](http://www.cs.washington.edu/education/courses/461/08au/projects/proj2.html)) and [wiki:EducationalAssignments/ChatServer Chat Server Assignment] focus on building applications that use the network. Students build network applications that run on top of Seattle and understand applications like HTTP and layering of network services. + * The [Web Server Assignment](Webserver.md) (based on [this assignment](http://www.cs.washington.edu/education/courses/461/08au/projects/proj2.html)) and [wiki:EducationalAssignments/ChatServer Chat Server Assignment] focus on building applications that use the network. Students build network applications that run on top of Seattle and understand applications like HTTP and layering of network services. - * The [wiki:EducationalAssignments/Chord Distributed Hash Table Assignment] has students build a peer-to-peer routing layer. Students begin by building an implementation of the popular DHT Chord. This will work well for routing messages over LAN topologies, but will fail in many WAN scenarios due to non-transitive connectivity. Students then use one-hop detour routing to route around failures. + * The [Distributed Hash Table Assignment](Chord.md) has students build a peer-to-peer routing layer. Students begin by building an implementation of the popular DHT Chord. This will work well for routing messages over LAN topologies, but will fail in many WAN scenarios due to non-transitive connectivity. Students then use one-hop detour routing to route around failures. -### Other Assignment Ideas +### Other Assignment Ideas A more complex set of assignments can also be designed for use with Seattle. The following assignment ideas are intended to show the scope of assignments Seattle can support. @@ -38,10 +38,10 @@ A more complex set of assignments can also be designed for use with Seattle. The Some cloud computing models provide an interface to a global, persistent store (like Amazon's S3 or Google's AppEngine). While Seattle provides local persistent storage, there is no preexisting global store. Such a system can be constructed from a set of nodes that cooperatively replicate and manage data. This project requires that student implementations operate correctly over LAN with multiple readers / writers and gracefully recovers from node failures. For a more difficult project, the global store can be required to run on globally distributed nodes. This extension teaches students to deal with high failure likelihood and to overcome non-transitive connectivity, both of which are typical problems for distributed systems deployed at global scale. -### Project Ideas +### Project Ideas Advanced undergraduate and graduate courses in networks and distributed systems can also use Seattle for a class project. Seattle project can involve everything from highly scalable, and complex distributed systems, to measurements of global Internet traffic patterns, to ubiquitous computing topics. The two project ideas below are examples of what such a graduate-level project would entail. - * [wiki:RepyMapReduce A MapReduce service] + * [A MapReduce service](../Applications/RepyMapReduce.md) The MapReduce algorithm harnesses the power of a multiple nodes to parallelize those computations that can be functionally decomposed into some number of map and reduce stages. Once completed, students would be able to use their implementations of MapReduce for practical compute jobs by simply supplying a map(), reduce(), and hash() methods to their MapReduce service. An example use of MapReduce is to implement a page-rank algorithm on a large subset of Wikipedia pages to find the “most popular” page. Using a database dump of Wikipedia’s pages, students can use one MapReduce pass to parse out all internal Wikipedia links and to generate a stem and leaf plot of all pages in Wikipedia. Using this list multiple MapReduce passes with the page rank algorithm will yield the comparative page rank for every Wikipedia page, creating a flat index that can be made accessible to users through a web server. @@ -59,12 +59,12 @@ Advanced undergraduate and graduate courses in networks and distributed systems -## Security / Operating Systems Assignments +## Security / Operating Systems Assignments - * The [wiki:EducationalAssignments/SecurityLayerPartOne Implementing Security Policies] introduces students to constructing a reference monitor. A student will implement a simple security policy that is meant to restrict access to files on a system. + * The [Implementing Security Policies](SecurityLayerPartOne.md) and [AB Storage Part One](ABStoragePartOne.md) introduces students to constructing a reference monitor. A student will implement a simple security policy that is meant to restrict access to files on a system. - * The [wiki:EducationalAssignments/SecurityLayerPartTwo Attacking Security Policies] follows up on the previous assignment by having students attack the security policies implemented in the previous assignment. Students get practical experience with how an attacker will try to violate security assumptions in a system. + * The [Attacking Security Policies](SecurityLayerPartTwo.md) and [AB Storage Part Two](ABStoragePartTwo.md)follows up on the previous assignment by having students attack the security policies implemented in the previous assignment. Students get practical experience with how an attacker will try to violate security assumptions in a system. (More to come!) From 2dab87bec6fe9f76d84e3903696a8fcfa0512ec3 Mon Sep 17 00:00:00 2001 From: Decidedly Gray Date: Wed, 25 Mar 2020 20:17:05 -0500 Subject: [PATCH 069/122] Update to fix markdown Fixed a small markdown typo --- Programming/PythonVsRepyV2.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Programming/PythonVsRepyV2.md b/Programming/PythonVsRepyV2.md index 3e980b46..7912ba3a 100644 --- a/Programming/PythonVsRepyV2.md +++ b/Programming/PythonVsRepyV2.md @@ -92,7 +92,7 @@ super, unichr, unicode, vars, yield, __import__ ---- #### sys.argv -Programs written in Python use `sys.argv``` to access arguments to +Programs written in Python use `sys.argv` to access arguments to the file. In RepyV2 the variable `callargs` behaves the same as `sys.argv[1:]` From 0f19b4e17c3d742589301d9529173f49cd3df4e2 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Thu, 1 Oct 2020 00:57:35 -0400 Subject: [PATCH 070/122] Update ABStoragePartTwo.md Clarified the security test >In order to do this, you will attempt to read, write, and append data without having the proper missions enabled This is only for the permissions assignment, here its just reading and writing invalid data A lot of places also have double spaces after sentences. --- EducationalAssignments/ABStoragePartTwo.md | 13 +++++-------- 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/EducationalAssignments/ABStoragePartTwo.md b/EducationalAssignments/ABStoragePartTwo.md index 13fdbbd9..402e6950 100644 --- a/EducationalAssignments/ABStoragePartTwo.md +++ b/EducationalAssignments/ABStoragePartTwo.md @@ -1,14 +1,12 @@ # Security layer testing and penetration -In this assignment you will learn how to attack a reference monitor. The reference monitor you will be testing uses the security layer framework (encasement library, etc.) for the Seattle testbed. It is possible to do this assignment separately, but it is recommended that this assignment be completed after [Part One](ABStoragePartOne.md). Either way you should already have a working security layer or access to one. Testing the security layer is done by running a series of test cases that an adversary may use to circumvent your system. This assignment is intended to prepare you for thinking about security paradigms in a functional way. The ideas of information, security and privacy have been embedded into the steps of this assignment. - - +In this assignment you will learn how to attack a reference monitor. The reference monitor you will be testing uses the security layer framework (encasement library, etc.) for the Seattle testbed. It is possible to do this assignment separately, but it is recommended that this assignment be completed after [Part One](ABStoragePartOne.md). Either way you should already have a working security layer or access to one. Testing the security layer is done by running a series of test cases that an adversary may use to circumvent your system. This assignment is intended to prepare you for thinking about security paradigms in a functional way. The ideas of information, security and privacy have been embedded into the steps of this assignment. ## Overview ---- -In this assignment you are a tester. You have been sent a bunch of reference monitors that need testing before they are deployed. Your job will be to ensure an attacker cannot circumvent these security layers. In order to do this, you will attempt to read, write, and append data without having the proper missions enabled. If you are able to do so, then the security layer is not secure. The future of the system depends on your ability to test code thoroughly! +In this assignment you are a tester. You have been sent a bunch of reference monitors that need testing before they are deployed. Your job will be to ensure an attacker cannot circumvent these security layers. In order to do this, you will attempt to read, write, and append invalid data. If you are able to do so, then the security layer is not secure. The future of the system depends on your ability to test code thoroughly! Three design paradigms are at work in this assignment: accuracy, efficiency, and security. @@ -20,14 +18,13 @@ Three design paradigms are at work in this assignment: accuracy, efficiency, and Within the context of this assignment these design paradigms translate to: - * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. For example, if an app tries to read data from the backup file, this must succeed as per normal and must not be blocked. All situations that are not described above must match that of the underlying API. + * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. These actions are defined in [ABStoragePartOne](./ABStoragePartOne.md) For example, if an app tries to read data from the backup file, this must succeed as per normal and must not be blocked. All situations that are not described above must match that of the underlying API. * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. For example, keeping a complete copy of every file on disk in memory would be forbidden. - * Security: The attacker should not be able to circumvent the security layer. For example, if the attacker can cause an invalid file to be saved, read the "write to" file, or can write to the backup file we read from, then the security is compromised. - -You will submit a zip file containing all of the tests you have created. You will gain points for every student's reference monitor you find a flaw in. It is good if multiple tests of yours break a student's reference monitor, but you gain the same number of tests whether one or more tests break the layer. + * Security: The attacker should not be able to circumvent the security layer. For example, if the security layer is correctly implemented but the attacker can still cause an invalid file to be saved, read the "write to" file, write to the backup file we read from, or read/write out of bounds, then the security is compromised. Note if preventing the aforementioned properties is part of the specification simply testing whether the actions can occur would merely be an Accuracy test. The keyword is circumvention. +You will submit a zip file containing all of the tests you have created. You will gain points for every student's reference monitor you find a flaw in. It is good if multiple tests of yours break a student's reference monitor, but you gain the same number of tests whether one or more tests break the layer. ## Prerequisites From 5547cf3eb9e6a4e68bcb84572ff0640a97fc47d3 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Thu, 8 Oct 2020 17:21:28 -0400 Subject: [PATCH 071/122] FIXED: Lost instruction I noticed somewhere along the way we lost the instruction to discard the backup and write to file after changes are saved to the original file. This also helps to clarify what the original file is, as if the a and b files are both discarded then they cannot be the original file --- EducationalAssignments/ABStoragePartOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/ABStoragePartOne.md b/EducationalAssignments/ABStoragePartOne.md index b7e14555..9515e71d 100644 --- a/EducationalAssignments/ABStoragePartOne.md +++ b/EducationalAssignments/ABStoragePartOne.md @@ -29,7 +29,7 @@ the first or last characters, then the file is considered invalid. Applications use ABopenfile() to create or open a file. Files are created by setting create=True when calling ABopenfile(), the reference monitor will create a valid backup file called filename.a and an empty -file we will write to called filename.b. When close() is called on the file, if both filename.a and filename.b are valid, the original file's data is replaced with the data of filename.b. If filename.b is not valid, no changes are made. +file we will write to called filename.b. When close() is called on the file, if both filename.a and filename.b are valid, the original file's data is replaced with the data of filename.b. If filename.b is not valid, the original file should take on the data of the backup a file. Afterward both the a and b file are discarded. Write test applications to ensure your reference monitor behaves properly in different cases and to test attacks against your monitor. From a1fa0ae636e2f05bdcd31f381f33c8b9cd1a7aa7 Mon Sep 17 00:00:00 2001 From: Victoria Zhong Date: Fri, 9 Oct 2020 01:53:35 -0400 Subject: [PATCH 072/122] Update ABStoragePartOne.md Attempted to make more clear what happens if filename.b is valid and what happens when it isn't. --- EducationalAssignments/ABStoragePartOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/ABStoragePartOne.md b/EducationalAssignments/ABStoragePartOne.md index 9515e71d..70dd80bb 100644 --- a/EducationalAssignments/ABStoragePartOne.md +++ b/EducationalAssignments/ABStoragePartOne.md @@ -29,7 +29,7 @@ the first or last characters, then the file is considered invalid. Applications use ABopenfile() to create or open a file. Files are created by setting create=True when calling ABopenfile(), the reference monitor will create a valid backup file called filename.a and an empty -file we will write to called filename.b. When close() is called on the file, if both filename.a and filename.b are valid, the original file's data is replaced with the data of filename.b. If filename.b is not valid, the original file should take on the data of the backup a file. Afterward both the a and b file are discarded. +file we will write to called filename.b. When close() is called on the file, if both filename.a and filename.b are valid, the original file's data is replaced with the data of filename.b. If filename.b is not valid, the original file should use the data of the backup filename.a file. Afterward both the filename.a and filename.b file should be deleted, and only the original file should remain. Write test applications to ensure your reference monitor behaves properly in different cases and to test attacks against your monitor. From 04b8bc3f706d770005632a908061c5aaad09256c Mon Sep 17 00:00:00 2001 From: Stephen Stenchever <30671118+sstenchever@users.noreply.github.com> Date: Sun, 11 Oct 2020 23:43:48 -0400 Subject: [PATCH 073/122] Update ABStoragePartTwo.md Test cases must be named: attackcase1.r2py, attackcase2.r2py, etc. in order to pass the auto-grader. Kevin mentioned it might be possible to add a description to the end such as attackcase1_file_length.r2py, but I haven't personally tested this. As far as I can tell there is no extra credit for this portion of the assignment, so this information should be removed. --- EducationalAssignments/ABStoragePartTwo.md | 12 +----------- 1 file changed, 1 insertion(+), 11 deletions(-) diff --git a/EducationalAssignments/ABStoragePartTwo.md b/EducationalAssignments/ABStoragePartTwo.md index 402e6950..19177378 100644 --- a/EducationalAssignments/ABStoragePartTwo.md +++ b/EducationalAssignments/ABStoragePartTwo.md @@ -172,15 +172,6 @@ And more! Remember a good security layer can't be broken by anyone! Which is a * Note that you should not assume that any files exist in your directory. You should create any files (e.g., testfile.txt) yourself in your test program. - - - -## Extra Credit ----- -Find bugs in the extra credit reference monitors given the altered threat model. You should include more test cases in the extra credit! - - - ## How to run your tests on many reference monitors ---- @@ -224,5 +215,4 @@ This will print out the name of each reference monitor before it starts executin ## What to turn in? ---- - * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must start with your netid in lowercase. For example: abc123_securitytest1.r2py abc123_goodaccuracytest.r2py are both valid names. - * Optionally turn in the test cases used to attack the extra credit reference monitors in a zip file. Note that in this case, you can expect that your code is run more than once. In the name of the file, say if it needs to be run multiple times. For example: abc123_run_twice_metadata_removal.r2py abc123_run_once_threading_hack.r2py. + * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must match the following format: attackcase1.r2py, attackcase2.r2py, etc. From 1e6ebee58a31ca3b6b516ee2d5be6bd8f2d7d18f Mon Sep 17 00:00:00 2001 From: sam-maverick <70863114+sam-maverick@users.noreply.github.com> Date: Thu, 12 Nov 2020 17:56:57 -0500 Subject: [PATCH 074/122] Update PythonVsRepyV2.md Added "with" statement clarification --- Programming/PythonVsRepyV2.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/Programming/PythonVsRepyV2.md b/Programming/PythonVsRepyV2.md index 7912ba3a..f07c0210 100644 --- a/Programming/PythonVsRepyV2.md +++ b/Programming/PythonVsRepyV2.md @@ -112,6 +112,11 @@ if callfunc == "initialize": main() ``` +#### with +Programs written in Python can use the `with` statement in combination +with `__enter__` and `__exit__` methods. However, these constructs are +not available in RepyV2. + ## Python Modules From 74ba202d3176fe9fa59b575b0e7c845c1674d34d Mon Sep 17 00:00:00 2001 From: sam-maverick <70863114+sam-maverick@users.noreply.github.com> Date: Thu, 12 Nov 2020 19:05:09 -0500 Subject: [PATCH 075/122] Update ABStoragePartOne.md Minor issue corrected --- EducationalAssignments/ABStoragePartOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/ABStoragePartOne.md b/EducationalAssignments/ABStoragePartOne.md index 70dd80bb..cd14e56f 100644 --- a/EducationalAssignments/ABStoragePartOne.md +++ b/EducationalAssignments/ABStoragePartOne.md @@ -27,7 +27,7 @@ valid file must start with the character 'S' and end with the character the first or last characters, then the file is considered invalid. Applications use ABopenfile() to create or open a file. Files are created by -setting create=True when calling ABopenfile(), the reference +setting create=True. When calling ABopenfile(), the reference monitor will create a valid backup file called filename.a and an empty file we will write to called filename.b. When close() is called on the file, if both filename.a and filename.b are valid, the original file's data is replaced with the data of filename.b. If filename.b is not valid, the original file should use the data of the backup filename.a file. Afterward both the filename.a and filename.b file should be deleted, and only the original file should remain. From ef321b84c4fc93aeacd587682a792f297cdf85fb Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Wed, 7 Sep 2022 06:01:15 +0800 Subject: [PATCH 076/122] Create LeftPadPartOne.md --- EducationalAssignments/LeftPadPartOne.md | 300 +++++++++++++++++++++++ 1 file changed, 300 insertions(+) create mode 100644 EducationalAssignments/LeftPadPartOne.md diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md new file mode 100644 index 00000000..200bc290 --- /dev/null +++ b/EducationalAssignments/LeftPadPartOne.md @@ -0,0 +1,300 @@ +# Implement a Defensive Security System + +This assignment will help you understand security mechanisms. You will be +guided through the steps of creating a reference monitor using the security +layer functionality in Repy V2. A reference monitor is an access control +concept that refers to an abstract machine that mediates all access to +objects by subjects. This can be used to allow, deny, or change the +behavior of any set of calls. While not a perfect way of validating your +reference monitor, it is useful to create test cases to see whether your +security layer will work as expected (the test cases may be turned in as +part of the next assignment). Please ask your instructor if test cases are available to you, some instructors may provide with test cases. + +This assignment is intended to reinforce concepts about access control and +reference monitors in a hands-on manner. + + + + +## Overview +---- +In this assignment you will create a security layer which will always left indent +after the '\n' newline character in a write that strictly appends to a file. By +"strictly appends", this means the first byte of the write is at the current end +of file (after the last previously written data). This is something that is +sometimes done by a document editors, when they believe you have ended a paragraph. +For this assignment, the '\n' character is treated as a newline and other characters +(notably '\r') are not treated specially. + +You should write test applications to ensure your reference monitor behaves properly +in different cases and to test attacks against your monitor. + +#### The Reference Monitor Must: +1. Behave identically to a non-sandboxed program [RepyV2 API calls](../Programming/RepyV2API.md), except: + * When writing to file using writeat() except when a write strictly appends to a file (as specified + in the next step). + * All other calls (writeat()s that do not strictly append, readat()s, etc.) must be performed as + they would in a non-sandboxed program! +2. If a writeat() strictly appends to a file, then: + a. If it contains no '\n', perform the writeat() operation normally (as a non-sandboxed program) + b. If it contains exactly one '\n' character, then perform a writeat() in the + same location, but with four space ' ' characters inserted before the '\n'. + c. If it contains more than one '\n' character, raise a RepyArgumentError exception +3. Not produce any errors or output for any reason except as mentioned above + * Normal operations should not be blocked or produce any output + * Invalid operations should not produce any output to the user +4. Call readat() everytime writeat() is called. This will be too slow and is forbidden. + + +Three design paradigms are at work in this assignment: accuracy, +efficiency, and security. + + * Accuracy: The security layer should only modify certain operations (a strictly +appending writeat() with one '\n') and raise an exception for certain other +actions (a strictly appending writeat() with more than one '\n'). All situations +that are not described above *must* match that of the underlying API. + + * Efficiency: The security layer should use a minimum number of resources, +so performance is not compromised. For example, you may not call readat() +everytime writeat() is called. It is permissable to call readat() upon fileopen(), +however. + + * Security: The attacker should not be able to circumvent the security +layer. For example, if the attacker can cause a non-strictly appending write to +have ' ' inserted before '\n' or can cause the reference monitor to incorrectly +error or hang, then the security is compromised. + + + +### Getting Python and RepyV2 + +Please refer to the [SeattleTestbed Build Instructions](../Contributing/BuildInstructions.md#prerequisites) +for details. + +Once you have built RepyV2 into a directory of your choice, change into that +directory. Use the command below in order to run your RepyV2 applications: + +```python2 repy.py restrictions.default encasementlib.r2py [security_layer].r2py [application].r2py``` + +(Replace '[security_layer].r2py' and '[application].r2py' by the names of the +security layers and application that you want to run.) + +In order to test whether or not these steps worked, please copy and paste +the code found below for the sample security layer and sample attack. + +If you got an error, please go through the troubleshooting section found below. + +### Troubleshooting Repy code +---- +If you can't get Repy files to run, some of the following common errors may +have occurred: + + * using `print` instead of `log`: + +Repy is a subset of Python, but its syntax is slightly different. For +example, Python's `print` statement cannot be used; Repy has `log` for +that. For a full list of acceptable syntax please see +[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md] + + * command line errors: + +**files are missing:** In the above command line call, you must have +`repy.py`, restrictions.default, encasementlib.r2py, the security layer and +the program you want to run in the current working directory. If any or +all of the above files are not in that directory then you will not be able +to run repy files. + + + + +### Tutorials for Repy and Python +---- +Now that you have Repy and Python, you may need a refresher on how to use +them. The following tutorials provide this information. + + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + + + +## Building the security layer +---- +The following program is a sample security layer, it is not complete and does not +handle all cases required by the API. Remember, you have no idea how the +attacker will try to penetrate your security layer, so it is important that +you leave nothing to chance! + + +### A basic (and inadequate) defense + +Time to start coding! Let's inspect a basic security layer. + +``` +""" +This security layer inadequately handles LeftPad writeat()s that strictly append + + + +Note: + This security layer uses encasementlib.r2py, restrictions.default, repy.py and Python + Also you need to give it an application to run. + python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py + + """ +TYPE="type" +ARGS="args" +RETURN="return" +EXCP="exceptions" +TARGET="target" +FUNC="func" +OBJC="objc" + +class LPFile(): + def __init__(self,filename,create): + # globals + mycontext['debug'] = False + if create: + self.LPfile = openfile(self.filename,create) + self.length = 0 + + + def writeat(self,data,offset): + if not offset == self.length: + # write the data and update the length (BUG?) + self.LPfile.writeat(data,offset) + self.length = offset + len(data) + + else: + + if '\n' not in data: + self.writeat(data,offset) + else: # bug? + loc = data.find('\n') + # bug? + self.writeat(data[:loc]+" "+data[loc:],offset) + + + def close(self): + self.LPfile.close() + + +def LPopenfile(filename, create): + return LPFile(filename,create) + + + + +# The code here sets up type checking and variable hiding for you. You +# should not need to change anything below here. +sec_file_def = {"obj-type":LPFile, + "name":"LPFile", + "writeat":{"type":"func","args":(str,(int,long)),"exceptions":Exception,"return":(int,type(None)),"target":LPFile.writeat}, + "readat":{"type":"func","args":((int,long,type(None)),(int,long)),"exceptions":Exception,"return":str,"target":LPFile.readat}, + "close":{"type":"func","args":None,"exceptions":None,"return":(bool,type(None)),"target":LPFile.close} + } + +CHILD_CONTEXT_DEF["openfile"] = {TYPE:OBJC,ARGS:(str,bool),EXCP:Exception,RETURN:sec_file_def,TARGET:LPopenfile} + +# Execute the user code +secure_dispatch_module() +``` + + + +### Testing your security layer +---- +In this part of the assignment you will pretend to be an attacker. Remember +the attacker's objective is to bypass the A/B restrictions or cause +the security layer to act in a disallowed manner. By understanding how the +attacker thinks, you will be able to write better security layers. + +An example of an attack is found below: + +``` +# clean up if the file exists. +if "testfile.txt" in listfiles(): + removefile("testfile.txt") + +myfile=openfile("testfile.txt",True) #Create a file + +myfile.writeat("12345678",0) # no difference, no '\n' + +myfile.writeat("Hi!",0) # writing early in the file + +myfile.writeat("Append!\nShould be indented!!!",8) # strictly appending... + +assert(' ' == myfile.readat(None,17)) # this location should contain a space... + +#Close the file +myfile.close() + + +``` + +If the reference monitor is correct, there should be no assertion failure... + +**Note:** All attacks should be written as Repy V2 files, using the .r2py extension. + +#### Choice of File Names +---- +Filenames may only be in the current directory and may only contain lowercase letters, numbers, the hyphen, underscore, and period characters. Also, filenames cannot be '.', '..', the blank string or start with a period. There is no concept of a directory or a folder in repy. Filenames must be no more than 120 characters long. + +### Running your security layer +---- +Finally, type the following commands at the terminal to run your security +layer with your attack program + +```python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py ``` + +Make sure you went through the "How to get RepyV2" section! + + +# Notes and Resources +---- + + * For a complete list of syntax in Repyv2 please visit: + * **[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md]** + + * The following link is an excellent source for information about security layers: **[https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf]** + + * **Note:** It is possible to add multiple security layers to Repy, this +may be useful for testing different mitigations separately. This is +done with the following command at the terminal: + +```python repy.py restrictions.default encasementlib.r2py [security_layer1].r2py [security_layer2].r2py [security_layer3].r2py [program].r2py``` + +**Your security layer must produce no output!! ** + + * In repy log replaces print from python. This may be helpful when +testing if Repy installed correctly. + + +# Extra Credit +---- +For extra credit, program that keeps all old versions of files and allows +read from any of them. Writing to any old file creates a new (empty) version +of that file. +Do not submit this code inside your assignment. Submit a separate copy for extra credit. + + + +# What to turn in? +---- + * Turn in a repy file called reference_monitor_[ netid ].r2py with all +letters in lowercase. + +* **Never raise unexpected errors or produce any output.** Your program +must produce no output when run normally. + + * For extra credit turn in a second repy file called extra_credit_[netid].r2py **You must turn in separate files for the normal assignment and extra credit** From 1926fce67247b2d85fb7b567815f7a14949aaada Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Wed, 7 Sep 2022 06:06:32 +0800 Subject: [PATCH 077/122] Update LeftPadPartOne.md --- EducationalAssignments/LeftPadPartOne.md | 18 ++++++++---------- 1 file changed, 8 insertions(+), 10 deletions(-) diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md index 200bc290..ffdb0670 100644 --- a/EducationalAssignments/LeftPadPartOne.md +++ b/EducationalAssignments/LeftPadPartOne.md @@ -30,20 +30,18 @@ You should write test applications to ensure your reference monitor behaves prop in different cases and to test attacks against your monitor. #### The Reference Monitor Must: -1. Behave identically to a non-sandboxed program [RepyV2 API calls](../Programming/RepyV2API.md), except: - * When writing to file using writeat() except when a write strictly appends to a file (as specified - in the next step). - * All other calls (writeat()s that do not strictly append, readat()s, etc.) must be performed as - they would in a non-sandboxed program! +1. Behave identically to a non-sandboxed program [RepyV2 API calls](../Programming/RepyV2API.md) for + all calls other than a writeat()s that strictly append. So other writeat()s, readat()s, etc. + must be performed as they would in a non-sandboxed program! 2. If a writeat() strictly appends to a file, then: - a. If it contains no '\n', perform the writeat() operation normally (as a non-sandboxed program) - b. If it contains exactly one '\n' character, then perform a writeat() in the - same location, but with four space ' ' characters inserted before the '\n'. - c. If it contains more than one '\n' character, raise a RepyArgumentError exception + * If it contains no '\n' characters, perform the writeat() operation normally (as a non-sandboxed program) + * If it contains exactly one '\n' character, then perform a writeat() that strictly appends, + but with four space ' ' characters inserted before the '\n'. + * If it contains more than one '\n' character, raise a RepyArgumentError exception 3. Not produce any errors or output for any reason except as mentioned above * Normal operations should not be blocked or produce any output * Invalid operations should not produce any output to the user -4. Call readat() everytime writeat() is called. This will be too slow and is forbidden. +4. Not call readat() everytime writeat() is called. This will be too slow and is forbidden. Three design paradigms are at work in this assignment: accuracy, From 8882a04ef0017ad17d31ad4847637e4de9520d1e Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Wed, 7 Sep 2022 21:49:14 +0800 Subject: [PATCH 078/122] Update LeftPadPartOne.md --- EducationalAssignments/LeftPadPartOne.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md index ffdb0670..b0f44ea0 100644 --- a/EducationalAssignments/LeftPadPartOne.md +++ b/EducationalAssignments/LeftPadPartOne.md @@ -162,10 +162,13 @@ class LPFile(): def __init__(self,filename,create): # globals mycontext['debug'] = False - if create: - self.LPfile = openfile(self.filename,create) - self.length = 0 + @ per-object vars + self.LPfile = openfile(filename,create) + self.length = 0 + def readat(self, bytes, offset): + # Read from the file using the sandbox's readat... + return self.file.readat(bytes, offset) def writeat(self,data,offset): if not offset == self.length: From 1c891db3bbb5889b5f0663f14de98b67321523c2 Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Wed, 7 Sep 2022 21:29:56 -0400 Subject: [PATCH 079/122] Update LeftPadPartOne.md Updated security layer and attack layer code. --- EducationalAssignments/LeftPadPartOne.md | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md index b0f44ea0..4e8a477f 100644 --- a/EducationalAssignments/LeftPadPartOne.md +++ b/EducationalAssignments/LeftPadPartOne.md @@ -162,13 +162,12 @@ class LPFile(): def __init__(self,filename,create): # globals mycontext['debug'] = False - @ per-object vars self.LPfile = openfile(filename,create) self.length = 0 def readat(self, bytes, offset): # Read from the file using the sandbox's readat... - return self.file.readat(bytes, offset) + return self.LPfile.readat(bytes, offset) def writeat(self,data,offset): if not offset == self.length: @@ -179,11 +178,11 @@ class LPFile(): else: if '\n' not in data: - self.writeat(data,offset) + self.LPfile.writeat(data,offset) else: # bug? loc = data.find('\n') # bug? - self.writeat(data[:loc]+" "+data[loc:],offset) + self.LPfile.writeat(data[:loc]+" "+data[loc:],offset) def close(self): @@ -235,7 +234,7 @@ myfile.writeat("Hi!",0) # writing early in the file myfile.writeat("Append!\nShould be indented!!!",8) # strictly appending... -assert(' ' == myfile.readat(None,17)) # this location should contain a space... +assert(' ' == myfile.readat(None,17)[0]) # this location should contain a space... #Close the file myfile.close() From 49a8ae2addcd08687f46f5ffbc621ce8e11e3d28 Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Wed, 7 Sep 2022 21:55:52 -0400 Subject: [PATCH 080/122] Update EducationalAssignments/LeftPadPartOne.md Yes, this is equivalent and much cleaner. Co-authored-by: Justin Cappos --- EducationalAssignments/LeftPadPartOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md index 4e8a477f..79dec532 100644 --- a/EducationalAssignments/LeftPadPartOne.md +++ b/EducationalAssignments/LeftPadPartOne.md @@ -234,7 +234,7 @@ myfile.writeat("Hi!",0) # writing early in the file myfile.writeat("Append!\nShould be indented!!!",8) # strictly appending... -assert(' ' == myfile.readat(None,17)[0]) # this location should contain a space... +assert(' ' == myfile.readat(1,17)) # this location should contain a space... #Close the file myfile.close() From d551475d0e3483839804ffa880120b89f78bcffa Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Mon, 26 Sep 2022 19:21:10 -0400 Subject: [PATCH 081/122] Update LeftPadPartOne.md Changed insertion of 4 space characters ' ' after '\n' instead of before. --- EducationalAssignments/LeftPadPartOne.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md index 79dec532..c19dc3c1 100644 --- a/EducationalAssignments/LeftPadPartOne.md +++ b/EducationalAssignments/LeftPadPartOne.md @@ -36,7 +36,7 @@ in different cases and to test attacks against your monitor. 2. If a writeat() strictly appends to a file, then: * If it contains no '\n' characters, perform the writeat() operation normally (as a non-sandboxed program) * If it contains exactly one '\n' character, then perform a writeat() that strictly appends, - but with four space ' ' characters inserted before the '\n'. + but with four space ' ' characters inserted after the '\n'. * If it contains more than one '\n' character, raise a RepyArgumentError exception 3. Not produce any errors or output for any reason except as mentioned above * Normal operations should not be blocked or produce any output @@ -59,7 +59,7 @@ however. * Security: The attacker should not be able to circumvent the security layer. For example, if the attacker can cause a non-strictly appending write to -have ' ' inserted before '\n' or can cause the reference monitor to incorrectly +have ' ' inserted after '\n' or can cause the reference monitor to incorrectly error or hang, then the security is compromised. From 1fcd00117db2a834d9daca65a0794714796b7da5 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 4 Oct 2022 16:02:41 -0400 Subject: [PATCH 082/122] Update LeftPadPartOne.md --- EducationalAssignments/LeftPadPartOne.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md index c19dc3c1..b648878d 100644 --- a/EducationalAssignments/LeftPadPartOne.md +++ b/EducationalAssignments/LeftPadPartOne.md @@ -80,7 +80,8 @@ security layers and application that you want to run.) In order to test whether or not these steps worked, please copy and paste the code found below for the sample security layer and sample attack. -If you got an error, please go through the troubleshooting section found below. +You should get an assertion error when running the test. If not, please go through the troubleshooting +section found below. ### Troubleshooting Repy code ---- From 2d53af3df968d53cecfcffe21470adbf23efb3ee Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Tue, 4 Oct 2022 20:07:26 -0400 Subject: [PATCH 083/122] Remove extra-credit --- EducationalAssignments/LeftPadPartOne.md | 11 ----------- 1 file changed, 11 deletions(-) diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md index b648878d..419806c0 100644 --- a/EducationalAssignments/LeftPadPartOne.md +++ b/EducationalAssignments/LeftPadPartOne.md @@ -281,15 +281,6 @@ done with the following command at the terminal: testing if Repy installed correctly. -# Extra Credit ----- -For extra credit, program that keeps all old versions of files and allows -read from any of them. Writing to any old file creates a new (empty) version -of that file. -Do not submit this code inside your assignment. Submit a separate copy for extra credit. - - - # What to turn in? ---- * Turn in a repy file called reference_monitor_[ netid ].r2py with all @@ -297,5 +288,3 @@ letters in lowercase. * **Never raise unexpected errors or produce any output.** Your program must produce no output when run normally. - - * For extra credit turn in a second repy file called extra_credit_[netid].r2py **You must turn in separate files for the normal assignment and extra credit** From 59799fad76aebf9664339cbb9a50055bc573fcbe Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Wed, 5 Oct 2022 21:06:11 -0400 Subject: [PATCH 084/122] Update BuildInstructions.md Add a line to explicitly mention using python2 for building as well. --- Contributing/BuildInstructions.md | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/Contributing/BuildInstructions.md b/Contributing/BuildInstructions.md index 041a8795..3cb7bc81 100644 --- a/Contributing/BuildInstructions.md +++ b/Contributing/BuildInstructions.md @@ -69,6 +69,13 @@ directory which will be created as a sub-directory inside your main repository. It takes its instructions from `config_initialize.txt` to fetch the dependencies. +Make sure that you are using python2 for initializing and building. To +check the python version being sourced you can do: + +```sh +$ python -V +``` + 3. To build a runnable component from the source dependencies, run the `scripts/build.py` script. You may supply it an optional target folder for the build (which must be created first). Name this folder as you like. From bbea305d97a95988a2318a85e9d05522e44c5725 Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Thu, 6 Oct 2022 14:43:26 -0400 Subject: [PATCH 085/122] Update BuildInstructions.md --- Contributing/BuildInstructions.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/Contributing/BuildInstructions.md b/Contributing/BuildInstructions.md index 3cb7bc81..3c072889 100644 --- a/Contributing/BuildInstructions.md +++ b/Contributing/BuildInstructions.md @@ -69,11 +69,11 @@ directory which will be created as a sub-directory inside your main repository. It takes its instructions from `config_initialize.txt` to fetch the dependencies. -Make sure that you are using python2 for initializing and building. To -check the python version being sourced you can do: +Make sure that you are using python 2.7 for initializing and building to +avoid any errors with building. To use python 2.7, you can do: ```sh -$ python -V +$ python2 initialize.py ``` 3. To build a runnable component from the source dependencies, run the From c0aeb8ad44c3bb900b642039a59c7f84beba9214 Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Thu, 6 Oct 2022 21:47:31 -0400 Subject: [PATCH 086/122] Update LeftPadPartOne.md Remove the line that says you can ask the instructor for test cases. --- EducationalAssignments/LeftPadPartOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md index 419806c0..a77a247b 100644 --- a/EducationalAssignments/LeftPadPartOne.md +++ b/EducationalAssignments/LeftPadPartOne.md @@ -8,7 +8,7 @@ objects by subjects. This can be used to allow, deny, or change the behavior of any set of calls. While not a perfect way of validating your reference monitor, it is useful to create test cases to see whether your security layer will work as expected (the test cases may be turned in as -part of the next assignment). Please ask your instructor if test cases are available to you, some instructors may provide with test cases. +part of the next assignment). This assignment is intended to reinforce concepts about access control and reference monitors in a hands-on manner. From 776c6660b18ff493d126c679ccc1e1950f5853ef Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Sun, 9 Oct 2022 23:42:25 -0400 Subject: [PATCH 087/122] Create LeftPadPartTwo.md --- EducationalAssignments/LeftPadPartTwo.md | 131 +++++++++++++++++++++++ 1 file changed, 131 insertions(+) create mode 100644 EducationalAssignments/LeftPadPartTwo.md diff --git a/EducationalAssignments/LeftPadPartTwo.md b/EducationalAssignments/LeftPadPartTwo.md new file mode 100644 index 00000000..3a254430 --- /dev/null +++ b/EducationalAssignments/LeftPadPartTwo.md @@ -0,0 +1,131 @@ + +# Security layer testing and penetration + +In this assignment you will learn how to attack a reference monitor. The reference monitor you will be testing uses the security layer framework (encasement library, etc.) for the Seattle testbed. It is possible to do this assignment separately, but it is recommended that this assignment be completed after [Part One](LeftPadPartOne.md). Either way you should already have a working security layer or access to one. Testing the security layer is done by running a series of test cases that an adversary may use to circumvent your system. This assignment is intended to prepare you for thinking about security paradigms in a functional way. The ideas of information, security and privacy have been embedded into the steps of this assignment. + + +## Overview +---- +In this assignment you are a tester. You have been sent a bunch of reference monitors that need testing before they are deployed. Your job will be to ensure an attacker cannot circumvent these security layers. In order to do this, you will attempt to write, and append invalid data. If you are able to do so, then the security layer is not secure. The future of the system depends on your ability to test code thoroughly! + +Three design paradigms are at work in this assignment: accuracy, efficiency, and security. + + * Accuracy: The security layer should only stop certain actions from being blocked. All other actions should be allowed. + + * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. + + * Security: The attacker should not be able to circumvent the security layer. + +Within the context of this assignment these design paradigms translate to: + + * Accuracy: The security layer should only modify certain operations (a strictly appending writeat() with one '\n') and raise an exception for certain other actions (a strictly appending writeat() with more than one '\n'). All situations that are not described above must match that of the underlying API. + + * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. For example, you may not call readat() everytime writeat() is called. It is permissible to call readat() upon fileopen(), however. + + * Security: The attacker should not be able to circumvent the security layer. For example, if the attacker can cause a non-strictly appending write to have ' ' inserted after '\n' or can cause the reference monitor to incorrectly error or hang, then the security is compromised. + +You will submit a zip file containing all of the tests you have created. You will gain points for every student's reference monitor you find a flaw in. It is good if multiple tests of yours break a student's reference monitor, but you gain the same number of tests whether one or more tests break the layer. + + +## Prerequisites + +This assignment assumes you have both the latest Python 2.7 and RepyV2 +installed on your computer. Please refer to the [SeattleTestbed Build Instructions](../Contributing/BuildInstructions.md#prerequisites) +for information on how to get them. + + +### Helpful links +---- +The following links will aid students in becoming comfortable with Python, Repy and seattle: + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + + +## Testing security layers +---- +### Hypothesis, test case, counter example + +The goal of a good tester is to test hypotheses. A hypothesis is just a scientific way of asking a question. The hypothesis of this assignment is "This security layer is well designed." The questions you will ask when running your test cases will always be the same + + * "Is this reference monitor secure?" + + * "Does this reference monitor hamper performance?" + + * "Does this reference monitor prevent actions that should be allowed?" + +Notice that these questions are parallels of the security paradigms: security, efficiency and accuracy, respectively. + +If we can find a case where the hypothesis is false, then the security layer is not secure. Such a case is referred to as a counter example. Hence all test cases should be designed to test for these three types of flaws. + + +#### Information on: Try, Except, Else, Finally +The try, except, else and finally statements are part of **exception handling**. For more information on exception handling please visit: + + * [http://docs.python.org/tutorial/errors.html] + * [http://wiki.python.org/moin/HandlingExceptions] + * [http://www.tutorialspoint.com/python/python_exceptions.htm] + +### Hints and Ideas for testing + +When writing your own tests it is important to test for a complete set of possible penetrations. Keep in mind, it only takes one test case to break through a security layer. Some of the things you may want to test for include: + + * threading + * multiple writes + +And more! Remember a good security layer can't be broken by anyone! Which is all part of the fun! It's about solving a puzzle. First you make the puzzle - write the security layer, then you solve the puzzle - try to bypass it. If your puzzle is "good enough", no one will be able to break it, no matter what. + + +## Notes and Resources +---- + + * The following link is an excellent source for information about security layers: https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf + + * In repy 'log' replaces 'print' from python. Many students find this to be a stumbling block. + + * Note that you should not assume that any files exist in your directory. You should create any files (e.g., testfile.txt) yourself in your test program. + +## How to run your tests on many reference monitors +---- + +Create a directory that the security layers will write their files into. You need to run repy with only access to this directory. You can write a test program that does `log(str(listfiles()))` to see if you are in the right place. + +Have all the reference monitors to test and the test cases inside the same directory where the repy.py file exists. + +* In the bash shell on Mac and Linux: +``` +for referencemonitor in reference_monitor_*; do for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` +* In the Command Prompt on Windows: +``` +FOR %r IN (reference_monitor_*) DO @FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a +``` +* In PowerShell on Windows: +``` +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the output from each program. Make sure that you replace `` with your NetID. + +If you want to spot the referencemonitor that failed during the test run, add echo the name of each referencemonitor before the inner loop, like so: + +* In the bash shell on Mac and Linux: +``` +for referencemonitor in reference_monitor_*; do echo $referencemonitor under test; for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` +* In the Command Prompt on Windows: +``` +FOR %r IN (reference_monitor_*) DO @(ECHO %r under test & FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a) +``` +* In PowerShell on Windows: +``` +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { Write-Host $referencemonitor.Name; foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the name of each reference monitor before it starts executing the testcases against it. + + +## What to turn in? +---- + + * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must match the following format: attackcase1.r2py, attackcase2.r2py, etc. From cf15823f70e17e0a11ad407a5a6cae19578894bc Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Mon, 10 Oct 2022 16:23:08 -0400 Subject: [PATCH 088/122] Update LeftPadPartTwo.md Added instruction to break tests into different files. --- EducationalAssignments/LeftPadPartTwo.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartTwo.md b/EducationalAssignments/LeftPadPartTwo.md index 3a254430..e89b2405 100644 --- a/EducationalAssignments/LeftPadPartTwo.md +++ b/EducationalAssignments/LeftPadPartTwo.md @@ -85,6 +85,8 @@ And more! Remember a good security layer can't be broken by anyone! Which is a * Note that you should not assume that any files exist in your directory. You should create any files (e.g., testfile.txt) yourself in your test program. + * It's important to note that if a test has a flaw in any part of it, the entire test will be considered invalid. So, it's advisable to break your tests into different files. + ## How to run your tests on many reference monitors ---- @@ -127,5 +129,4 @@ This will print out the name of each reference monitor before it starts executin ## What to turn in? ---- - * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must match the following format: attackcase1.r2py, attackcase2.r2py, etc. From 6b20eae64d4c04f49c3cc46c8e7bd85b7baa4bfb Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Tue, 18 Oct 2022 16:16:50 -0400 Subject: [PATCH 089/122] Update LeftPadPartOne.md add exception for close() --- EducationalAssignments/LeftPadPartOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartOne.md b/EducationalAssignments/LeftPadPartOne.md index a77a247b..05802d12 100644 --- a/EducationalAssignments/LeftPadPartOne.md +++ b/EducationalAssignments/LeftPadPartOne.md @@ -202,7 +202,7 @@ sec_file_def = {"obj-type":LPFile, "name":"LPFile", "writeat":{"type":"func","args":(str,(int,long)),"exceptions":Exception,"return":(int,type(None)),"target":LPFile.writeat}, "readat":{"type":"func","args":((int,long,type(None)),(int,long)),"exceptions":Exception,"return":str,"target":LPFile.readat}, - "close":{"type":"func","args":None,"exceptions":None,"return":(bool,type(None)),"target":LPFile.close} + "close":{"type":"func","args":None,"exceptions":Exception,"return":(bool,type(None)),"target":LPFile.close} } CHILD_CONTEXT_DEF["openfile"] = {TYPE:OBJC,ARGS:(str,bool),EXCP:Exception,RETURN:sec_file_def,TARGET:LPopenfile} From 1af9d359b86e2e9560f91b152899bb8c01744644 Mon Sep 17 00:00:00 2001 From: Devansh-Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Sun, 23 Oct 2022 19:33:02 -0400 Subject: [PATCH 090/122] Update LeftPadPartTwo.md --- EducationalAssignments/LeftPadPartTwo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartTwo.md b/EducationalAssignments/LeftPadPartTwo.md index e89b2405..5863f71d 100644 --- a/EducationalAssignments/LeftPadPartTwo.md +++ b/EducationalAssignments/LeftPadPartTwo.md @@ -129,4 +129,4 @@ This will print out the name of each reference monitor before it starts executin ## What to turn in? ---- - * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must match the following format: attackcase1.r2py, attackcase2.r2py, etc. + * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must match the following format: [ net_id ]_attackcase1.r2py, [ net_id ]_attackcase2.r2py, etc. From 771a0762f1319b9388596045a12367bfa4602a1d Mon Sep 17 00:00:00 2001 From: Devansh Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Tue, 25 Oct 2022 16:21:10 -0400 Subject: [PATCH 091/122] Create LeftPadPartThree.md Please give extra attention to the code review part and do tell me if anything should be removed or any extra bug should be mentioned. --- EducationalAssignments/LeftPadPartThree.md | 67 ++++++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100644 EducationalAssignments/LeftPadPartThree.md diff --git a/EducationalAssignments/LeftPadPartThree.md b/EducationalAssignments/LeftPadPartThree.md new file mode 100644 index 00000000..65fd888c --- /dev/null +++ b/EducationalAssignments/LeftPadPartThree.md @@ -0,0 +1,67 @@ + +# Fixing your security layer + +In this assignment you will analyze the bugs in your security layer from [Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartOne.md) and fix them. You may want to use test cases from [Part Two](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartTwo.md) to help identify these bugs. Finally, you will write a report discussing the different classes of bugs that your code had, and why. + + + + +## Overview +---- +In this assignment you are fixing your reference monitor. You have been sent a bunch of test programs that try to compromise your reference monitor. Your job will be to determine where your reference monitor failed, and fix it to ensure an attacker cannot circumvent the security layer. + + + +## Common Bugs + * The reference monitor needs to track the state of the information on disk, but cannot re-read it for every access (due to efficiency concerns). A common mistake is when the attacker can cause the reference monitor’s state to diverge from the underlying system’s state, especially in error conditions. + + * Time-of-check-to-time-of-use (TOCTTOU) bugs and other types of race conditions are a fairly common oversight for students. Some students write test cases that attempt to trigger a race condition to exploit these problems. This can result in essentially any sort of attack, even infinite loops in the reference monitor in some cases. + + * Some reference monitors inappropriately share state for different files. For example, there may be a global set of state that is used to store the first two bytes. By opening multiple files, an attacker may be able to overwrite this state and cause security and accuracy issues. + + * In rare cases, a student’s reference monitor may inappropriately retain state for a file. For example, an attacker may create a file, write some data, then close and delete the file. If the attacker recreates a file with the same name, the state should be cleared. + + * Many reference monitors had accuracy or security bugs as a result of not properly following the instructions. + + + + +## Example : Sample Implementation + +``` +class LPFile(): + def __init__(self,filename,create): + # globals + mycontext['debug'] = False + self.LPfile = openfile(filename,create) + self.length = 0 + + def readat(self, bytes, offset): + # Read from the file using the sandbox's readat... + return self.LPfile.readat(bytes, offset) + + def writeat(self,data,offset): + if not offset == self.length: + # write the data and update the length (BUG?) + self.LPfile.writeat(data,offset) + self.length = offset + len(data) + + else: + + if '\n' not in data: + self.LPfile.writeat(data,offset) + else: # bug? + loc = data.find('\n') + # bug? + self.LPfile.writeat(data[:loc]+" "+data[loc:],offset) +``` + +### Code Analysis +Let's analyze one of the bugs in this implementation. According to the specifications in [Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartOne.md), it should return RepyArgumentError if there are two '\n' which isn't implemented in this code, morever, this code inserts spaces before '\n', as a result you can't see the indentation. Another bug is that it should first update the length and write. Furthermore, this code doesn't check if len(data) is greater than self.length when offset is less then self.length. It also assumes self.lenght = 0 which means that the file would be empty. Moreover, This example doesn't use any locks so you can introduce race conditions and perform a variety of attacks. + + +## What to turn in? +---- + + * Turn in the reference monitor that you have fixed. The name of the reference monitor should be in the form of reference_monitor_[netId].r2py +All letters must be lowercase. From cf0ffc7da36b33151a4879c26434d5a945571d61 Mon Sep 17 00:00:00 2001 From: Devansh Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Tue, 25 Oct 2022 16:22:53 -0400 Subject: [PATCH 092/122] Update LeftPadPartThree.md fixed typo --- EducationalAssignments/LeftPadPartThree.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartThree.md b/EducationalAssignments/LeftPadPartThree.md index 65fd888c..158e4bcf 100644 --- a/EducationalAssignments/LeftPadPartThree.md +++ b/EducationalAssignments/LeftPadPartThree.md @@ -57,7 +57,7 @@ class LPFile(): ``` ### Code Analysis -Let's analyze one of the bugs in this implementation. According to the specifications in [Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartOne.md), it should return RepyArgumentError if there are two '\n' which isn't implemented in this code, morever, this code inserts spaces before '\n', as a result you can't see the indentation. Another bug is that it should first update the length and write. Furthermore, this code doesn't check if len(data) is greater than self.length when offset is less then self.length. It also assumes self.lenght = 0 which means that the file would be empty. Moreover, This example doesn't use any locks so you can introduce race conditions and perform a variety of attacks. +Let's analyze one of the bugs in this implementation. According to the specifications in [Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartOne.md), it should return RepyArgumentError if there are two '\n' which isn't implemented in this code, moreover, this code inserts spaces before '\n', as a result you can't see the indentation. Another bug is that it should first update the length and write. Furthermore, this code doesn't check if len(data) is greater than self.length when offset is less then self.length. It also assumes self.lenght = 0 which means that the file would be empty. Moreover, This example doesn't use any locks so you can introduce race conditions and perform a variety of attacks. ## What to turn in? From 69c4e342c7980260f72d7b8ee62244afe000e6c0 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Tue, 25 Oct 2022 23:39:43 -0400 Subject: [PATCH 093/122] Update EducationalAssignments/LeftPadPartThree.md --- EducationalAssignments/LeftPadPartThree.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartThree.md b/EducationalAssignments/LeftPadPartThree.md index 158e4bcf..65aa02c4 100644 --- a/EducationalAssignments/LeftPadPartThree.md +++ b/EducationalAssignments/LeftPadPartThree.md @@ -57,7 +57,7 @@ class LPFile(): ``` ### Code Analysis -Let's analyze one of the bugs in this implementation. According to the specifications in [Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartOne.md), it should return RepyArgumentError if there are two '\n' which isn't implemented in this code, moreover, this code inserts spaces before '\n', as a result you can't see the indentation. Another bug is that it should first update the length and write. Furthermore, this code doesn't check if len(data) is greater than self.length when offset is less then self.length. It also assumes self.lenght = 0 which means that the file would be empty. Moreover, This example doesn't use any locks so you can introduce race conditions and perform a variety of attacks. +Let's analyze one of the bugs in this implementation. According to the specifications in [Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartOne.md), it should return RepyArgumentError if there are two '\n' which isn't implemented in this code, moreover, this code inserts spaces before '\n', as a result you can't see the indentation. Another bug is that it should first update the length and write. Furthermore, this code doesn't check if len(data) is greater than self.length when offset is less than self.length. It also assumes self.length = 0 which means that the file would be empty. Moreover, This example doesn't use any locks so you can introduce race conditions and perform a variety of attacks. ## What to turn in? From f65937ca6360417e65d947ec1c55c20401167fb7 Mon Sep 17 00:00:00 2001 From: Devansh Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Fri, 28 Oct 2022 15:53:11 -0400 Subject: [PATCH 094/122] Update LeftPadPartTwo.md --- EducationalAssignments/LeftPadPartTwo.md | 1 + 1 file changed, 1 insertion(+) diff --git a/EducationalAssignments/LeftPadPartTwo.md b/EducationalAssignments/LeftPadPartTwo.md index 5863f71d..ae7185ea 100644 --- a/EducationalAssignments/LeftPadPartTwo.md +++ b/EducationalAssignments/LeftPadPartTwo.md @@ -129,4 +129,5 @@ This will print out the name of each reference monitor before it starts executin ## What to turn in? ---- + * Never raise unexpected errors or produce any output. Your program must produce no output when run normally. * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must match the following format: [ net_id ]_attackcase1.r2py, [ net_id ]_attackcase2.r2py, etc. From f37d6b4d0f25879410602e28013a2b4c7d7c72ed Mon Sep 17 00:00:00 2001 From: Devansh Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Fri, 28 Oct 2022 15:55:41 -0400 Subject: [PATCH 095/122] Update LeftPadPartThree.md --- EducationalAssignments/LeftPadPartThree.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartThree.md b/EducationalAssignments/LeftPadPartThree.md index 65aa02c4..c045e675 100644 --- a/EducationalAssignments/LeftPadPartThree.md +++ b/EducationalAssignments/LeftPadPartThree.md @@ -62,6 +62,6 @@ Let's analyze one of the bugs in this implementation. According to the specific ## What to turn in? ---- - + * Never raise unexpected errors or produce any output. Your program must produce no output when run normally. * Turn in the reference monitor that you have fixed. The name of the reference monitor should be in the form of reference_monitor_[netId].r2py All letters must be lowercase. From debae4ffe46bd0de0f51c52cd877f0238be9a4d2 Mon Sep 17 00:00:00 2001 From: Devansh Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Fri, 28 Oct 2022 15:56:43 -0400 Subject: [PATCH 096/122] Update LeftPadPartTwo.md --- EducationalAssignments/LeftPadPartTwo.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/LeftPadPartTwo.md b/EducationalAssignments/LeftPadPartTwo.md index ae7185ea..d8166ada 100644 --- a/EducationalAssignments/LeftPadPartTwo.md +++ b/EducationalAssignments/LeftPadPartTwo.md @@ -129,5 +129,5 @@ This will print out the name of each reference monitor before it starts executin ## What to turn in? ---- - * Never raise unexpected errors or produce any output. Your program must produce no output when run normally. + * Never raise unexpected errors or produce any output. Your attack must produce no output when run normally. * Turn in the test cases used to attack a given reference monitor in a zip file. The name of each testcase must match the following format: [ net_id ]_attackcase1.r2py, [ net_id ]_attackcase2.r2py, etc. From 345c78832bbc0e72caf0d5760413efbf5d570933 Mon Sep 17 00:00:00 2001 From: Devansh Patel <45604637+Devansh-Patel@users.noreply.github.com> Date: Thu, 3 Nov 2022 21:10:21 -0400 Subject: [PATCH 097/122] Remove the line mentioning report --- EducationalAssignments/LeftPadPartThree.md | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/EducationalAssignments/LeftPadPartThree.md b/EducationalAssignments/LeftPadPartThree.md index c045e675..1361c0a0 100644 --- a/EducationalAssignments/LeftPadPartThree.md +++ b/EducationalAssignments/LeftPadPartThree.md @@ -1,8 +1,7 @@ # Fixing your security layer -In this assignment you will analyze the bugs in your security layer from [Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartOne.md) and fix them. You may want to use test cases from [Part Two](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartTwo.md) to help identify these bugs. Finally, you will write a report discussing the different classes of bugs that your code had, and why. - +In this assignment you will analyze the bugs in your security layer from [Part One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartOne.md) and fix them. You may want to use test cases from [Part Two](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/LeftPadPartTwo.md) to help identify these bugs. From 9268409406d43081d879271ed2255bdd4a315a98 Mon Sep 17 00:00:00 2001 From: Venu Vardhan Reddy Tekula Date: Wed, 27 Sep 2023 06:54:45 -0700 Subject: [PATCH 098/122] Create UndoPartOne.md Co-authored-by: Justin Cappos --- EducationalAssignments/UndoPartOne.md | 303 ++++++++++++++++++++++++++ 1 file changed, 303 insertions(+) create mode 100644 EducationalAssignments/UndoPartOne.md diff --git a/EducationalAssignments/UndoPartOne.md b/EducationalAssignments/UndoPartOne.md new file mode 100644 index 00000000..39bf140e --- /dev/null +++ b/EducationalAssignments/UndoPartOne.md @@ -0,0 +1,303 @@ +# Implement a Defensive Security System + +This assignment will help you understand security mechanisms. You will be guided +through the steps of creating a reference monitor using the security layer +functionality in Repy V2. A reference monitor is an access control concept that +refers to an abstract machine that mediates all access to objects by subjects. +This can be used to allow, deny, or change the behavior of any set of calls. +While not a perfect way of validating your reference monitor, it is useful to +create test cases to see whether your security layer will work as expected (the +test cases may be turned in as part of the next assignment). + +This assignment is intended to reinforce concepts about access control and +reference monitors in a hands-on manner. + + +## Overview + +In this assignment, you will implement a defense monitor to oversee file +operations in Repy. The monitor will enhance the default Repy behavior by adding +an undo functionality for writes. This defense monitor will ensure that software +follows certain rules and guidelines, preventing potential unauthorized actions. + +The focus is on monitoring file write operations and allowing the last write to +be reverted. This undo capability mimics common document editors that let you +reverse your most recent edit. + +You should write test applications to ensure your reference monitor behaves +properly in different cases and to test attacks against your monitor. + + +### Specifications: + +1. Your defense monitor should incorporate all the standard file operation + methods, with an added method named `undo`. +2. A `writeat` operation will not immediately write its changes to the file. The + changes will only be permanent after either of the following occur: + - A subsequent valid `writeat` operation is executed. + - The file is closed. +3. The `undo` operation will reverse the changes of the last valid `writeat` + operation, making it as if it didn't happen. It's crucial to note that only + the most recent operation can be undone. If there haven't been any `writeat` + operations, the `undo` has no effect. If several `writeat` operations are + consecutively executed, only the changes from the last valid `writeat` can be + undone. +4. Aside from the ability to be undone and potentially delayed writes, all + `writeat` operations should behave the same way as they do in the RepyV2 API. + + +Three design paradigms are at work in this assignment: accuracy, efficiency, and +security. + + * Accuracy: The defense monitor should precisely and consistently manage file +operations. Only specific operations, such as `writeat`, should have their +behavior modified. All situations that are not described above *must* match that +of the underlying API. + + * Efficiency: The security layer should use a minimum number of resources, so +performance is not compromised. For example, you may not call `readat()` +everytime `writeat()` is called. It is permissable to call `readat()` upon +`fileopen()`, however. + + * Security: The defense layer should be robust against tampering and +circumvention. Attackers must not be able to bypass, disable, or exploit the +defense monitor's enhanced behaviors, ensuring the integrity and intended +functionality of file operations. + + +### Getting Python and RepyV2 + +Please refer to the [SeattleTestbed Build +Instructions](../Contributing/BuildInstructions.md#prerequisites) for details. + +Once you have built RepyV2 into a directory of your choice, change into that +directory. Use the command below in order to run your RepyV2 applications: + +```bash +python2 repy.py restrictions.default encasementlib.r2py [security_layer].r2py [application].r2py +``` + +(Replace `[security_layer].r2py` and `[application].r2py` by the names of the +security layers and application that you want to run.) + +In order to test whether or not these steps worked, please copy and paste the +code found below for the sample security layer and sample attack. + +You should get an assertion error when running the test. If not, please go +through the troubleshooting section found below. + + +### Troubleshooting Repy code + +If you can't get Repy files to run, some of the following common errors may have +occurred: + + * using `print` instead of `log`: + +Repy is a subset of Python, but its syntax is slightly different. For example, +Python's `print` statement cannot be used; Repy has `log` for that. For a full +list of acceptable syntax please see +[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md] + + * command line errors: + +**files are missing:** In the above command line call, you must have `repy.py`, +restrictions.default, encasementlib.r2py, the security layer and the program you +want to run in the current working directory. If any or all of the above files +are not in that directory then you will not be able to run repy files. + + + + +### Tutorials for Repy and Python + +Now that you have Repy and Python, you may need a refresher on how to use them. +The following tutorials provide this information. + + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + + + +## Building the security layer + +The following program is a sample security layer, it is not complete and does +not handle all cases required by the API. Remember, you have no idea how the +attacker will try to penetrate your security layer, so it is important that you +leave nothing to chance! + + +### A basic (and inadequate) defense + +Time to start coding! Let's inspect a basic security layer. + +```py +""" +This security layer inadequately handles the undo functionality + +Note: + This security layer uses encasementlib.r2py, restrictions.default, repy.py and Python + Also you need to give it an application to run. + python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py + +""" +TYPE = "type" +ARGS = "args" +RETURN = "return" +EXCP = "exceptions" +TARGET = "target" +FUNC = "func" +OBJC = "objc" + + +class LPFile(): + def __init__(self,filename,create): + # globals + mycontext['debug'] = False + self.LPfile = openfile(filename,create) + self.pending_data = "" + self.pending_offset = 0 + + def readat(self, bytes, offset): + # Read from the file using the sandbox's readat... + return self.LPfile.readat(bytes, offset) + + def writeat(self,data,offset): + self.LPfile.writeat(data, offset) + self.pending_data += data + self.pending_offset = offset + len(data) + + def undo(self): + self.LPfile.writeat("", self.pending_offset) + + def close(self): + self.LPfile.close() + +def LPopenfile(filename, create): + return LPFile(filename, create) + +# The code here sets up type checking and variable hiding for you. +# You should not need to change anything below here. +sec_file_def = { + "obj-type": LPFile, + "name": "LPFile", + "writeat": {"type": "func","args": (str, (int, long)),"exceptions": Exception,"return": (int, type(None)),"target": LPFile.writeat}, + "readat": {"type": "func","args": ((int, long, type(None)), (int, long)),"exceptions": Exception,"return": str,"target": LPFile.readat}, + "undo": {"type": "func","args": None,"exceptions": (),"return": type(None),"target": LPFile.undo}, + "close": {"type": "func","args": None,"exceptions": Exception,"return": (bool, type(None)),"target": LPFile.close} +} + +CHILD_CONTEXT_DEF["openfile"] = { + TYPE: OBJC, + ARGS: (str, bool), + EXCP: Exception, + RETURN: sec_file_def, + TARGET: LPopenfile +} + +# Execute the user code +secure_dispatch_module() +``` + + +### Testing your security layer + +In this part of the assignment you will pretend to be an attacker. Remember the +attacker's objective is to bypass the A/B restrictions or cause the security +layer to act in a disallowed manner. By understanding how the attacker thinks, +you will be able to write better security layers. + +An example of an attack is found below: + +```py +# clean up if the file exists. +if "testfile.txt" in listfiles(): + removefile("testfile.txt") + +# create a file +myfile=openfile("testfile.txt",True) + +# intial write to the file +myfile.writeat("12345678",0) + +# attempt to overwrite the beginning of the file +myfile.writeat("Hi!",0) + +# now, undo the previous write +myfile.undo() + +# the contents should still be "12345678" as the overwrite was undone +assert("12345678" == myfile.readat(8,0)) + +# close the file +myfile.close() +``` + +If the reference monitor is correct, there should be no assertion failure. + +**Note:** All attacks should be written as Repy V2 files, using the `.r2py` +extension. + + +### Choice of File Names + +Filenames may only be in the current directory and may only contain lowercase +letters, numbers, the hyphen, underscore, and period characters. Also, filenames +cannot be '.', '..', the blank string or start with a period. There is no +concept of a directory or a folder in repy. Filenames must be no more than 120 +characters long. + + +### Running your security layer + +Finally, type the following commands at the terminal to run your security layer +with your attack program + +```bash +python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py +``` + +Make sure you went through the "How to get RepyV2" section! + + +# Notes and Resources + + * For a complete list of syntax in Repyv2 please visit: + **[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md]** + + * The following link is an excellent source for information about security + layers: + **[https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf]** + + * **Note:** It is possible to add multiple security layers to Repy, this may be +useful for testing different mitigations separately. This is done with the +following command at the terminal: + +```bash +python repy.py restrictions.default encasementlib.r2py [security_layer1].r2py [security_layer2].r2py [security_layer3].r2py [program].r2py +``` + + * **Your security layer must produce no output!!** + + * In repy log replaces print from python. This may be helpful when testing if +Repy installed correctly. + + +# What to turn in? + + * Turn in a repy file called `reference_monitor_[ netid ].r2py` with all +letters in lowercase. + +* **Never raise unexpected errors or produce any output.** Your program must +produce no output when run normally. From 7a136ce1e644e665bebcf06db8672e7b963bebc8 Mon Sep 17 00:00:00 2001 From: Venu Vardhan Reddy Tekula Date: Sun, 1 Oct 2023 17:51:22 -0700 Subject: [PATCH 099/122] Update the inadequate defense layer --- EducationalAssignments/UndoPartOne.md | 34 +++++++++++++-------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/EducationalAssignments/UndoPartOne.md b/EducationalAssignments/UndoPartOne.md index 39bf140e..2a1d8dec 100644 --- a/EducationalAssignments/UndoPartOne.md +++ b/EducationalAssignments/UndoPartOne.md @@ -83,7 +83,7 @@ security layers and application that you want to run.) In order to test whether or not these steps worked, please copy and paste the code found below for the sample security layer and sample attack. -You should get an assertion error when running the test. If not, please go +You should get an `RepyArgumentError` when running the test. If not, please go through the troubleshooting section found below. @@ -129,7 +129,6 @@ The following tutorials provide this information. * List of [RepyV2 API calls](../Programming/RepyV2API.md) - ## Building the security layer The following program is a sample security layer, it is not complete and does @@ -140,7 +139,7 @@ leave nothing to chance! ### A basic (and inadequate) defense -Time to start coding! Let's inspect a basic security layer. +Time to start coding! Let's inspect a basic security layer. ```py """ @@ -162,24 +161,25 @@ OBJC = "objc" class LPFile(): - def __init__(self,filename,create): + def __init__(self, filename, create): # globals - mycontext['debug'] = False - self.LPfile = openfile(filename,create) - self.pending_data = "" - self.pending_offset = 0 + mycontext['debug'] = False + self.LPfile = openfile(filename, create) + self.pending_data = None + self.pending_offset = None def readat(self, bytes, offset): # Read from the file using the sandbox's readat... return self.LPfile.readat(bytes, offset) - def writeat(self,data,offset): - self.LPfile.writeat(data, offset) - self.pending_data += data - self.pending_offset = offset + len(data) + def writeat(self, data, offset): + self.LPfile.writeat(self.pending_data, self.pending_offset) + self.pending_data = data + self.pending_offset = offset def undo(self): - self.LPfile.writeat("", self.pending_offset) + self.pending_data = data + self.pending_offset = offset def close(self): self.LPfile.close() @@ -192,10 +192,10 @@ def LPopenfile(filename, create): sec_file_def = { "obj-type": LPFile, "name": "LPFile", - "writeat": {"type": "func","args": (str, (int, long)),"exceptions": Exception,"return": (int, type(None)),"target": LPFile.writeat}, - "readat": {"type": "func","args": ((int, long, type(None)), (int, long)),"exceptions": Exception,"return": str,"target": LPFile.readat}, - "undo": {"type": "func","args": None,"exceptions": (),"return": type(None),"target": LPFile.undo}, - "close": {"type": "func","args": None,"exceptions": Exception,"return": (bool, type(None)),"target": LPFile.close} + "writeat": {"type": "func", "args": (str, (int, long)), "exceptions": Exception, "return": (int, type(None)), "target": LPFile.writeat}, + "readat": {"type": "func", "args": ((int, long, type(None)), (int, long)), "exceptions": Exception, "return": str, "target": LPFile.readat}, + "undo": {"type": "func", "args": None, "exceptions": (), "return": type(None), "target": LPFile.undo}, + "close": {"type": "func", "args": None, "exceptions": Exception, "return": (bool, type(None)), "target": LPFile.close} } CHILD_CONTEXT_DEF["openfile"] = { From 0c05fa92318071b083342c7d1cf4aaa67469adf5 Mon Sep 17 00:00:00 2001 From: Venu Vardhan Reddy Tekula Date: Mon, 2 Oct 2023 15:30:18 -0400 Subject: [PATCH 100/122] Fix issues in the sample defense layer --- EducationalAssignments/UndoPartOne.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/EducationalAssignments/UndoPartOne.md b/EducationalAssignments/UndoPartOne.md index 2a1d8dec..eea3a93a 100644 --- a/EducationalAssignments/UndoPartOne.md +++ b/EducationalAssignments/UndoPartOne.md @@ -178,8 +178,8 @@ class LPFile(): self.pending_offset = offset def undo(self): - self.pending_data = data - self.pending_offset = offset + self.pending_data = None + self.pending_offset = None def close(self): self.LPfile.close() @@ -194,7 +194,7 @@ sec_file_def = { "name": "LPFile", "writeat": {"type": "func", "args": (str, (int, long)), "exceptions": Exception, "return": (int, type(None)), "target": LPFile.writeat}, "readat": {"type": "func", "args": ((int, long, type(None)), (int, long)), "exceptions": Exception, "return": str, "target": LPFile.readat}, - "undo": {"type": "func", "args": None, "exceptions": (), "return": type(None), "target": LPFile.undo}, + "undo": {"type": "func", "args": None, "exceptions": None, "return": type(None), "target": LPFile.undo}, "close": {"type": "func", "args": None, "exceptions": Exception, "return": (bool, type(None)), "target": LPFile.close} } @@ -244,7 +244,7 @@ assert("12345678" == myfile.readat(8,0)) myfile.close() ``` -If the reference monitor is correct, there should be no assertion failure. +If the reference monitor is correct, there should be no `RepyArgumentError`. **Note:** All attacks should be written as Repy V2 files, using the `.r2py` extension. From b29ad8d3cdbe8fcb7487f47925d04224bea6bf3b Mon Sep 17 00:00:00 2001 From: Venu Vardhan Reddy Tekula Date: Wed, 11 Oct 2023 14:33:50 -0400 Subject: [PATCH 101/122] Create UndoPartTwo.md --- EducationalAssignments/UndoPartTwo.md | 192 ++++++++++++++++++++++++++ 1 file changed, 192 insertions(+) create mode 100644 EducationalAssignments/UndoPartTwo.md diff --git a/EducationalAssignments/UndoPartTwo.md b/EducationalAssignments/UndoPartTwo.md new file mode 100644 index 00000000..f53b1810 --- /dev/null +++ b/EducationalAssignments/UndoPartTwo.md @@ -0,0 +1,192 @@ +# Security layer testing and penetration + +In this assignment you will learn how to attack a reference monitor. The +reference monitor you will be testing uses the security layer framework +(encasement library, etc.) for the Seattle testbed. It is possible to do this +assignment separately, but it is recommended that this assignment be completed +after [Part One](./UndoPartOne.md). Either way you should already have a working +security layer or access to one. Testing the security layer is done by running a +series of test cases that an adversary may use to circumvent your system. This +assignment is intended to prepare you for thinking about security paradigms in a +functional way. The ideas of information, security and privacy have been +embedded into the steps of this assignment. + + +## Overview + +In this assignment you are a tester. You have been sent a bunch of reference +monitors that need testing before they are deployed. Your job will be to ensure +an attacker cannot circumvent these security layers. In order to do this, you +will attempt to write, and append invalid data. If you are able to do so, then +the security layer is not secure. The future of the system depends on your +ability to test code thoroughly! + +Three design paradigms are at work in this assignment: accuracy, efficiency, and +security. + + * Accuracy: The security layer should only stop certain actions from being + blocked. All other actions should be allowed. + + * Efficiency: The security layer should use a minimum number of resources, so + performance is not compromised. + + * Security: The attacker should not be able to circumvent the security layer. + + +Within the context of this assignment these design paradigms translate to: + + * Accuracy: The defense monitor should precisely and consistently manage file +operations. Only specific operations, such as `writeat`, should have their +behavior modified. All situations that are not described in the specifications +*must* match that of the underlying API. + + * Efficiency: The security layer should use a minimum number of resources, so +performance is not compromised. For example, you may not call `readat()` +everytime `writeat()` is called. It is permissable to call `readat()` upon +`fileopen()`, however. + + * Security: The defense layer should be robust against tampering and +circumvention. Attackers must not be able to bypass, disable, or exploit the +defense monitor's enhanced behaviors, ensuring the integrity and intended +functionality of file operations. + +You will submit a zip file containing all of the tests you have created. You +will gain points for every student's reference monitor you find a flaw in. It is +good if multiple tests of yours break a student's reference monitor, but you +gain the same number of tests whether one or more tests break the layer. + + +## Prerequisites + +This assignment assumes you have both the latest Python 2.7 and RepyV2 installed +on your computer. Please refer to the [SeattleTestbed Build +Instructions](../Contributing/BuildInstructions.md#prerequisites) for +information on how to get them. + + +### Helpful links + +The following links will aid students in becoming comfortable with Python, Repy +and seattle: + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + + +## Testing security layers + +### Hypothesis, test case, counter example + +The goal of a good tester is to test hypotheses. A hypothesis is just a +scientific way of asking a question. The hypothesis of this assignment is "This +security layer is well designed." The questions you will ask when running your +test cases will always be the same + + * "Is this reference monitor secure?" + + * "Does this reference monitor hamper performance?" + + * "Does this reference monitor prevent actions that should be allowed?" + +Notice that these questions are parallels of the security paradigms: security, +efficiency and accuracy, respectively. + +If we can find a case where the hypothesis is false, then the security layer is +not secure. Such a case is referred to as a counter example. Hence all test +cases should be designed to test for these three types of flaws. + +#### Information on: Try, Except, Else, Finally + +The try, except, else and finally statements are part of **exception handling**. +For more information on exception handling please visit: + + * [http://docs.python.org/tutorial/errors.html] + * [http://wiki.python.org/moin/HandlingExceptions] + * [http://www.tutorialspoint.com/python/python_exceptions.htm] + +### Hints and Ideas for testing + +When writing your own tests it is important to test for a complete set of +possible penetrations. Keep in mind, it only takes one test case to break +through a security layer. Some of the things you may want to test for include: + + * threading + * multiple writes + +And more! Remember a good security layer can't be broken by anyone! Which is +all part of the fun! It's about solving a puzzle. First you make the puzzle - +write the security layer, then you solve the puzzle - try to bypass it. If your +puzzle is "good enough", no one will be able to break it, no matter what. + + +## Notes and Resources + + * The following link is an excellent source for information about security + layers: https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf + + * In repy `log` replaces `print` from python. Many students find this to be a + stumbling block. + + * Note that you should not assume that any files exist in your directory. You + should create any files (e.g., `testfile.txt`) yourself in your test program. + + * It's important to note that if a test has a flaw in any part of it, the + entire test will be considered invalid. So, it's advisable to break your + tests into different files. + +## How to run your tests on many reference monitors + +Create a directory that the security layers will write their files into. You +need to run repy with only access to this directory. You can write a test +program that does `log(str(listfiles()))` to see if you are in the right place. + +Have all the reference monitors to test and the test cases inside the same +directory where the `repy.py` file exists. + +* In the bash shell on Mac and Linux: +```bash +for referencemonitor in reference_monitor_*; do for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` + +* In the Command Prompt on Windows: +```cmd +FOR %r IN (reference_monitor_*) DO @FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a +``` + +* In PowerShell on Windows: +```powershell +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the output from each program. Make sure that you replace +`` with your NetID. + +If you want to spot the reference monitor that failed during the test run, add +echo the name of each reference monitor before the inner loop, like so: + +* In the bash shell on Mac and Linux: +```bash +for referencemonitor in reference_monitor_*; do echo $referencemonitor under test; for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` + +* In the Command Prompt on Windows: +```cmd +FOR %r IN (reference_monitor_*) DO @(ECHO %r under test & FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a) +``` + +* In PowerShell on Windows: +```powershell +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { Write-Host $referencemonitor.Name; foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the name of each reference monitor before it starts +executing the test cases against it. + + +## What to turn in? + + * Never raise unexpected errors or produce any output. Your attack must produce + no output when run normally. + * Turn in the test cases used to attack a given reference monitor in a zip + file. The name of each testcase must match the following format: `[ net_id + ]_attackcase1.r2py`, `[ net_id ]_attackcase2.r2py`, etc. From 77b1d90e336b5bee84830dce03e558ca85341930 Mon Sep 17 00:00:00 2001 From: Venu Vardhan Reddy Tekula Date: Fri, 13 Oct 2023 13:37:13 -0400 Subject: [PATCH 102/122] Update UndoPartTwo.md --- EducationalAssignments/UndoPartTwo.md | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/EducationalAssignments/UndoPartTwo.md b/EducationalAssignments/UndoPartTwo.md index f53b1810..ffd79a90 100644 --- a/EducationalAssignments/UndoPartTwo.md +++ b/EducationalAssignments/UndoPartTwo.md @@ -119,6 +119,14 @@ write the security layer, then you solve the puzzle - try to bypass it. If your puzzle is "good enough", no one will be able to break it, no matter what. +> **Note**: When creating tests, it's best practice to separate them into +> different files. If there is a flaw in any part of a test, it can invalidate +> the entire thing. Breaking tests into separate files helps isolate issues and +> prevent one flawed test from compromising others. This modular approach makes +> debugging easier and allows you to more efficiently identify and address +> problems. + + ## Notes and Resources * The following link is an excellent source for information about security From ab1f77aae747b407508b678eff68b791842bedfb Mon Sep 17 00:00:00 2001 From: Venu Vardhan Reddy Tekula Date: Fri, 13 Oct 2023 17:28:42 -0400 Subject: [PATCH 103/122] Update UndoPartTwo.md --- EducationalAssignments/UndoPartTwo.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/EducationalAssignments/UndoPartTwo.md b/EducationalAssignments/UndoPartTwo.md index ffd79a90..d649ce14 100644 --- a/EducationalAssignments/UndoPartTwo.md +++ b/EducationalAssignments/UndoPartTwo.md @@ -119,12 +119,12 @@ write the security layer, then you solve the puzzle - try to bypass it. If your puzzle is "good enough", no one will be able to break it, no matter what. -> **Note**: When creating tests, it's best practice to separate them into -> different files. If there is a flaw in any part of a test, it can invalidate -> the entire thing. Breaking tests into separate files helps isolate issues and -> prevent one flawed test from compromising others. This modular approach makes -> debugging easier and allows you to more efficiently identify and address -> problems. +> Note: When creating tests, the best practice is to separate them into +> different files. Each test file should focus on testing one specific scenario +> or functionality. If there is a flaw in any part of a test file, we will +> discard the entire test file. This modular approach helps isolate issues - if +> there is a flaw in one test file, it will not affect or invalidate the other +> test files, since each file is independent and testing distinct functionality. ## Notes and Resources From e1e8ac664d27e8c63b7b1aee7f18aaa15452f5bc Mon Sep 17 00:00:00 2001 From: Venu Vardhan Reddy Tekula Date: Mon, 23 Oct 2023 14:23:09 -0400 Subject: [PATCH 104/122] Create UndoPartThree.md --- EducationalAssignments/UndoPartThree.md | 105 ++++++++++++++++++++++++ 1 file changed, 105 insertions(+) create mode 100644 EducationalAssignments/UndoPartThree.md diff --git a/EducationalAssignments/UndoPartThree.md b/EducationalAssignments/UndoPartThree.md new file mode 100644 index 00000000..ff45ce0d --- /dev/null +++ b/EducationalAssignments/UndoPartThree.md @@ -0,0 +1,105 @@ +# Fixing your security layer + +In this assignment you will analyze the bugs in your security layer from [Part +One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/UndoPartOne.md) +and fix them. You may want to use test cases from [Part +Two](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/UndoPartTwo.md) +to help identify these bugs. + + +## Overview + +In this assignment you are fixing your reference monitor. You have been given a +bunch of test programs that try to compromise your reference monitor. Your job +will be to determine where your reference monitor failed, and fix it to ensure +an attacker cannot circumvent the security layer. + + +## Common Bugs + + * The reference monitor needs to track the state of the information on disk, + but cannot re-read it for every access (due to efficiency concerns). A common + mistake is when the attacker can cause the reference monitor’s state to + diverge from the underlying system’s state, especially in error conditions. + + * Time-of-check-to-time-of-use (TOCTTOU) bugs and other types of race + conditions are a fairly common oversight for students. Some students write + test cases that attempt to trigger a race condition to exploit these + problems. This can result in essentially any sort of attack, even infinite + loops in the reference monitor in some cases. + + * Some reference monitors inappropriately share state for different files. For + example, there may be a global set of state that is used to store the first + two bytes. By opening multiple files, an attacker may be able to overwrite + this state and cause security and accuracy issues. + + * In rare cases, a student’s reference monitor may inappropriately retain state + for a file. For example, an attacker may create a file, write some data, then + close and delete the file. If the attacker recreates a file with the same + name, the state should be cleared. + + * Many reference monitors had accuracy or security bugs as a result of not + properly following the instructions. + + +## Example : Sample Implementation + +```py +class LPFile(): + def __init__(self, filename, create): + # globals + mycontext['debug'] = False + self.LPfile = openfile(filename, create) + self.pending_data = None + self.pending_offset = None + + def readat(self, bytes, offset): + # Read from the file using the sandbox's readat... + return self.LPfile.readat(bytes, offset) + + def writeat(self, data, offset): + # bug ? + self.LPfile.writeat(self.pending_data, self.pending_offset) + self.pending_data = data + self.pending_offset = offset + + def undo(self): + # bug ? + self.pending_data = None + self.pending_offset = None + + def close(self): + # bug ? + self.LPfile.close() +``` + +### Code Analysis + +Let's analyze some of the bugs in this implementation. + +According to the specifications in [Part +One](https://github.com/SeattleTestbed/docs/blob/master/EducationalAssignments/UndoPartOne.md), +the `writeat` function is designed to work such that the provided data is not +immediately written to the file. Instead, it should be stored in a "pending" +state. Here, the `writeat` writes to the file immediately without any form of +delay. Another bug is that on closing the file, the function doesn't check or +commit any `pending_data` to the file. Thus, if there was any `pending_data` +from the last `writeat` operation, it will be lost and not written to the file. + +There's no error handling for scenarios like writing with a negative offset, +writing beyond the end of the file, etc. Such error checks are essential to +prevent data corruption and unexpected behavior. Furthermore, there's no +mechanism to update or manage the current length of the file. This can cause +problems, especially if you're trying to prevent writing beyond the end of the +file or if you're trying to append data correctly. Moreover, this example +doesn't use any locks so you can introduce race conditions and perform a variety +of attacks. + + +## What to turn in? + + * Never raise unexpected errors or produce any output. Your program must + produce no output when run normally. + * Turn in the reference monitor that you have fixed. The name of the reference +monitor should be in the form of `reference_monitor_[ netid ].r2py`. All letters +must be lowercase. From 7b1d704f82a35380712f125463a6e0304d6d454d Mon Sep 17 00:00:00 2001 From: Venu Vardhan Reddy Tekula Date: Fri, 13 Oct 2023 20:17:39 -0400 Subject: [PATCH 105/122] Add `auto_grader.py` script This commit adds the automated grading script for repy_v2 from https://github.com/Hooshangi/Grading-script. It also adds some documentation. Co-authored-by: Sara Hooshangi --- Scripts/README.md | 67 ++++++++++++ Scripts/auto_grader.py | 235 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 302 insertions(+) create mode 100644 Scripts/README.md create mode 100644 Scripts/auto_grader.py diff --git a/Scripts/README.md b/Scripts/README.md new file mode 100644 index 00000000..8f317e0c --- /dev/null +++ b/Scripts/README.md @@ -0,0 +1,67 @@ +# `auto_grader.py` +> **Automated Grading for Repy V2 Assignments** + +`auto_grader.py` is a Python-based tool designed to streamline the grading +process for defense and attack programs written in [Repy +V2](https://github.com/SeattleTestbed/repy_v2). By automatically running each +attack against every defense, it produces comprehensive results in a CSV format. + +## Description +With `auto_grader.py`, instructors can effortlessly assess the efficacy of +students' defense mechanisms in the face of potential attacks. For every attack +that succeeds in compromising a defense, the resulting CSV will register a `1`; +otherwise, it will display a `0`. An attack is considered successful if it +generates an output or raises an error, denoting the failure of the defense +layer. It also handles timeouts, ensuring that the script does not hang in the +event of an infinite loop. + +## Prerequisites +- Python 2.7 installed on your machine. +- Repy V2 environment setup. +- Copy the required files (mentioned below) in the script's directory + - `repy.py` + - `restrictions.default` + - `wrapper.r2py` + - `encasementlib.r2py` + +## Usage +```bash +python auto_grader.py defense_folder_path attack_folder_path temp_target_folder_path +``` +where: +- `defense_folder_path` is the path to the folder containing defense programs. +- `attack_folder_path` is the path to the folder containing attack programs. +- `temp_target_folder_path` is the path to the temporary target folder. + +## Naming Conventions +- Attack programs should be named as: "`[studentid]_attackcase[number].r2py`". +- Defense programs should start with the name + "`reference_monitor_[studentid].r2py`". + +For example, if the student id is `abc123`, the defense program should be named +as `reference_monitor_abc123.r2py` and the attack program should be named as +`abc123_attackcase1.r2py`, `abc123_attackcase2.r2py`, etc. + +## Output +Two CSV files are generated: +1. `All_Attacks_matrix.csv`: Contains the result of every attack program against + each defense. +2. `All_Students_matrix.csv`: Indicates which students successfully attacked a + defense. + +## Notes +- Students are instructed to generate output or raise an error in their attack + program only when they successfully compromise the security layer. +- Ensure the correct environment, naming conventions, and directory structures + are adhered to for successful script execution. + +## Contributing +For modifications, improvements, or any issues, please open a pull request or +issue. + +## Credits +`auto_grader.py` is the brainchild of +[@Hooshangi](https://github.com/Hooshangi). + +For further details, potential contributions, or to view the code, visit +[Hooshangi/Grading-script](https://github.com/Hooshangi/Grading-script). diff --git a/Scripts/auto_grader.py b/Scripts/auto_grader.py new file mode 100644 index 00000000..c0f0dfc3 --- /dev/null +++ b/Scripts/auto_grader.py @@ -0,0 +1,235 @@ +""" + + auto_grader.py + + + Module which takes all defense programs and attack programs in Repy V2 and creates + csv file describing test result of every attack program against each defense program. + Under the attack program column, it shows one if attack was able to compromise security layer + and 0 otherwise. + Students submitting attack and defense programs are instructed to log output or raise error + in their attack program only when the security layer gets failed. In this program we check which + attack program is producing output or error. If so that means the attack is successful and that + security layer is compromised. + + + -All defense programs should be in a folder and all attack programs in another folder. + auto_grader, repy.py ,restrictions.default, wrapper.r2py, encasementlib.r2py. + -Create a temporary target folder inside the directory. + + python auto_grader.py defense_folder_path attack_folder_path temp_target_folder_path + +""" + + + + +import os +import glob +import subprocess +import shutil +import csv +import time +import signal # using SIGKILL +import sys + + +path_DefenseFolder = sys.argv[1] +path_AttackFolder = sys.argv[2] + + +# Paths to temp folder where the attack is run +path_TempFolder = sys.argv[3] + +#part of filename to look for +attack_ext = "*.r2py" #extension of attack programs will be r2py +def_ext = "reference*" #students are instructed to name their defense program starting with reference + + + + +def get_student_ID(attack_fnlist): + ''' Given a list of the total attack files, get a list of all of the + student IDs. This assumes that the file names contains _ + ''' + + studentset = set() + for attackfn in attack_fnlist: + # I assume everything before the first _ is the student ID. I will + # raise an exception for a file like foo.repy... + + if len(attackfn.split('_')) == 0: + raise ValueError('File name "'+attackfn+'" should contain an underscore!') + + studentname = attackfn.split('_')[0] + studentset.add(studentname) + + return sorted(list(studentset)) + + +def get_student_attack_code(studentname, attack_fnlist): + ''' This will return only the attack code from a specific student.''' + + thisstudentattacks = [] + for thisattack in attack_fnlist: + # need the underscore to stop bob from matching bobby's tests. + if thisattack.startswith(studentname+'_'): + thisstudentattacks.append(thisattack) + + + return thisstudentattacks + + +def check_if_student_attacks_succeed(student_attackfn, defensefn): + ''' Returns a list of any code for a student that produce stderr or stdout. + An empty list is returned if none were successful + otherwise''' + + successfulattacks = [] + + for attackfn in student_attackfn: + if did_this_attack_succeed(attackfn, defensefn): + successfulattacks.append(attackfn) + + return successfulattacks + + + + +def did_this_attack_succeed(attackFilename, defenseFilename): + ''' Returns True if the attack produces stderr or stdout, + False otherwise + ''' + + timeout=30 + + os.mkdir(path_TempFolder) # make a temp folder + os.chdir(path_TempFolder) # cd to temp folder at this point + + shutil.copy(path_DefenseFolder + '/' + defenseFilename, path_TempFolder + '/' + defenseFilename) + shutil.copy(path_AttackFolder + '/' + attackFilename, path_TempFolder + '/' + attackFilename) + shutil.copy('../wrapper.r2py', path_TempFolder + '/wrapper.r2py') + start = time.time() + + pobj = subprocess.Popen( + ['python', '../repy.py', '--stop=Repy_stop_this', '../restrictions.default', '../encasementlib.r2py', defenseFilename, + attackFilename], stdout=subprocess.PIPE, stderr=subprocess.PIPE) + + # NOT A BUG: Note that this will not get all of the stdout or stderr because we are polling for completion. + # Substantial output may cause the program to block, which is a very bad thing... in most cases. + # Since output / errput is failure and timeout is failure, we're actually okay with it here. + while pobj.poll() is None: + time.sleep(0.1) + now = time.time() + if now - start > timeout: + # Signal for the repyVM to stop (due to the --stop option) + file("Repy_stop_this","w").close() + # wait for it to stop... + pobj.wait() + stdout = "timeout" + stderr = "timeout" + break + else: + (stdout, stderr) = pobj.communicate() + + + os.chdir(path_AttackFolder) # go back to attack folder + shutil.rmtree(path_TempFolder) #remove the temp directory + + + if stdout != '' or stderr !='': + return True + else: + return False + + + +def row_builder(success_list,original_list,defenseFilename): + + num_success=len(success_list) + + matrix_row=[0]*(len(original_list)) + matrix_row[0]=defenseFilename + + for element in range(num_success): + + for row_num in range(len(original_list)): + if original_list[row_num]==success_list[element]: + matrix_row[row_num]=1 # if an attack is successful, put a 1 in the matrix + + return matrix_row + + + + + + +def main(): + + # If the temp folder is left over from last time, remove it. + if os.path.exists(path_TempFolder): + shutil.rmtree(path_TempFolder) + + os.chdir(path_DefenseFolder) # cd to defense monitor folder + defense_fnlist =glob.glob(def_ext) + + os.chdir(path_AttackFolder) # cd to attack folder + attack_fnlist=glob.glob(attack_ext) + + studentIDlist = get_student_ID(attack_fnlist) + print 'number of students in the course', len(studentIDlist) + + header_attackmatrix=attack_fnlist + header_attackmatrix.insert(0,'All attack files-->') + + header_studentmatrix=studentIDlist + header_studentmatrix.insert(0,'All students -->') + + resultFile1 = open("All_Attacks_matrix.csv",'wb') + wr_allattacks = csv.writer(resultFile1) + wr_allattacks.writerow(header_attackmatrix) + + resultFile2 = open("All_Students_matrix.csv",'wb') + wr_students = csv.writer(resultFile2) + wr_students.writerow(header_studentmatrix) + + + for defenseFilename in defense_fnlist: + collection_successful_attacks=list() + collection_successful_students=list() + + print 'This is defense file --->', defenseFilename + + for attackingstudent in studentIDlist: + + # get just this student's attacks + student_attackfns = get_student_attack_code(attackingstudent,attack_fnlist) + successfulattacks = check_if_student_attacks_succeed(student_attackfns,defenseFilename) + + + if successfulattacks!=[]: + + print defenseFilename,'---attacked by--- ', attackingstudent,'----->', successfulattacks + collection_successful_attacks=collection_successful_attacks+successfulattacks + collection_successful_students.append(attackingstudent) + + #print 'successful attacks',collection_successful_attacks + row_all_attacks=row_builder(collection_successful_attacks,attack_fnlist,defenseFilename) + row_students=row_builder(collection_successful_students,studentIDlist,defenseFilename) + + wr_allattacks.writerow(row_all_attacks) + wr_students.writerow(row_students) + + print 'row completed with students',row_students + + + + + +if __name__ == "__main__": + main() + + + + + From 632d3719dfa0e8a3bce3f1ae620f031efd784c92 Mon Sep 17 00:00:00 2001 From: Venu Vardhan Reddy Tekula Date: Sun, 19 Nov 2023 23:22:45 -0500 Subject: [PATCH 106/122] Update UndoParOne to add more clarifications --- EducationalAssignments/UndoPartOne.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/EducationalAssignments/UndoPartOne.md b/EducationalAssignments/UndoPartOne.md index eea3a93a..fb389da1 100644 --- a/EducationalAssignments/UndoPartOne.md +++ b/EducationalAssignments/UndoPartOne.md @@ -33,7 +33,7 @@ properly in different cases and to test attacks against your monitor. 1. Your defense monitor should incorporate all the standard file operation methods, with an added method named `undo`. 2. A `writeat` operation will not immediately write its changes to the file. The - changes will only be permanent after either of the following occur: + changes will only be permanent (commit) after either of the following occur: - A subsequent valid `writeat` operation is executed. - The file is closed. 3. The `undo` operation will reverse the changes of the last valid `writeat` @@ -44,7 +44,10 @@ properly in different cases and to test attacks against your monitor. undone. 4. Aside from the ability to be undone and potentially delayed writes, all `writeat` operations should behave the same way as they do in the RepyV2 API. - + You are expected to keep track of the offset, to make sure the attack is not + writing past the EOF. +5. The `undo` operation raises `FileClosedError` if the file is already closed. +6. The `readat` cannot read data that has not been committed to the file. Three design paradigms are at work in this assignment: accuracy, efficiency, and security. From 713347ad342625aaa210e1578af264d8265b209d Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Mon, 27 Nov 2023 15:09:59 -0500 Subject: [PATCH 107/122] Update EducationalAssignments/UndoPartOne.md --- EducationalAssignments/UndoPartOne.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/EducationalAssignments/UndoPartOne.md b/EducationalAssignments/UndoPartOne.md index fb389da1..05f5b438 100644 --- a/EducationalAssignments/UndoPartOne.md +++ b/EducationalAssignments/UndoPartOne.md @@ -44,8 +44,6 @@ properly in different cases and to test attacks against your monitor. undone. 4. Aside from the ability to be undone and potentially delayed writes, all `writeat` operations should behave the same way as they do in the RepyV2 API. - You are expected to keep track of the offset, to make sure the attack is not - writing past the EOF. 5. The `undo` operation raises `FileClosedError` if the file is already closed. 6. The `readat` cannot read data that has not been committed to the file. From 33cbd7ee34821761c57a841589a9e260024661b9 Mon Sep 17 00:00:00 2001 From: Justin Cappos Date: Mon, 27 Nov 2023 15:11:03 -0500 Subject: [PATCH 108/122] Update EducationalAssignments/UndoPartOne.md --- EducationalAssignments/UndoPartOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/UndoPartOne.md b/EducationalAssignments/UndoPartOne.md index 05f5b438..358089b7 100644 --- a/EducationalAssignments/UndoPartOne.md +++ b/EducationalAssignments/UndoPartOne.md @@ -45,7 +45,7 @@ properly in different cases and to test attacks against your monitor. 4. Aside from the ability to be undone and potentially delayed writes, all `writeat` operations should behave the same way as they do in the RepyV2 API. 5. The `undo` operation raises `FileClosedError` if the file is already closed. -6. The `readat` cannot read data that has not been committed to the file. +6. `readat` cannot read data that has not been committed to the file. Three design paradigms are at work in this assignment: accuracy, efficiency, and security. From 13d2bd0fb702bedd85a8c63a3db798d679bc580a Mon Sep 17 00:00:00 2001 From: Rajath Reghunath Date: Mon, 16 Sep 2024 17:45:47 -0400 Subject: [PATCH 109/122] Add draft for ISP Fall 2024 Assignment 2.1 --- EducationalAssignments/DefaultPartOne.md | 302 +++++++++++++++++++++++ 1 file changed, 302 insertions(+) create mode 100644 EducationalAssignments/DefaultPartOne.md diff --git a/EducationalAssignments/DefaultPartOne.md b/EducationalAssignments/DefaultPartOne.md new file mode 100644 index 00000000..a80b3a36 --- /dev/null +++ b/EducationalAssignments/DefaultPartOne.md @@ -0,0 +1,302 @@ +# Implement a Defensive Security System + +This assignment will help you understand security mechanisms. You will be guided +through the steps of creating a reference monitor using the security layer +functionality in Repy V2. A reference monitor is an access control concept that +refers to an abstract machine that mediates all access to objects by subjects. +This can be used to allow, deny, or change the behavior of any set of calls. +While not a perfect way of validating your reference monitor, it is useful to +create test cases to see whether your security layer will work as expected (the +test cases may be turned in as part of the next assignment). + +This assignment is intended to reinforce concepts about access control and +reference monitors in a hands-on manner. + + +## Overview + +In this assignment, you will implement a defense monitor to oversee file +operations in Repy. The monitor will enhance the default Repy behavior by adding +support for a default file to be used when files are opened without creating them first. +This defense monitor will ensure that software follows certain rules and guidelines, preventing potential unauthorized actions. + +You should write test applications to ensure your reference monitor behaves +properly in different cases and to test attacks against your monitor. + + +### Specifications: + +1. Your defense monitor should incorporate all the standard file operation methods, from opening a file, reading and writing to it, to closing and deleting it. + +2. In addition, a specially named file - `default` - shall be used as a template when opening files without creating them first. +Eg: Calling `openfile('foo', True)` should create and open a new empty file called `foo` (assuming it's not present already). However, calling `openfile('foo', False)` should create a new file using `default` as the template. If `default` doesn't exist, throw the relevant error (`FileNotFoundError`). + +3. If default is created, written to or deleted, then all closed files that were previously created gets deleted. Any files that are already open must be left unchanged. + +Three design paradigms are at work in this assignment: accuracy, efficiency, and +security. + +* Accuracy: The defense monitor should precisely and consistently manage file +operations. Only specific operations, such as `openfile()`, should have their +behavior modified. All situations that are not described above *must* match that +of the underlying API. + +* Efficiency: The security layer should use a minimum number of resources, so +performance is not compromised. For example, you may not call `readat()` +everytime `writeat()` is called. It is permissable to call `readat()` upon +`fileopen()`, however. + + * Security: The defense layer should be robust against tampering and +circumvention. Attackers must not be able to bypass, disable, or exploit the +defense monitor's enhanced behaviors, ensuring the integrity and intended +functionality of file operations. + + +### Getting Python and RepyV2 + +Please refer to the [SeattleTestbed Build +Instructions](../Contributing/BuildInstructions.md#prerequisites) for details. + +Once you have built RepyV2 into a directory of your choice, change into that +directory. Use the command below in order to run your RepyV2 applications: + +```bash +python2 repy.py restrictions.default encasementlib.r2py [security_layer].r2py [application].r2py +``` + +(Replace `[security_layer].r2py` and `[application].r2py` by the names of the +security layers and application that you want to run.) + +In order to test whether or not these steps worked, please copy and paste the +code found below for the sample security layer and sample attack. + +You should get a `FileNotFoundError` when running the test. If not, please go +through the troubleshooting section found below. + + +### Troubleshooting Repy code + +If you can't get Repy files to run, some of the following common errors may have +occurred: + + * using `print` instead of `log`: + +Repy is a subset of Python, but its syntax is slightly different. For example, +Python's `print` statement cannot be used; Repy has `log` for that. For a full +list of acceptable syntax please see +[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md] + + * command line errors: + +**files are missing:** In the above command line call, you must have `repy.py`, +restrictions.default, encasementlib.r2py, the security layer and the program you +want to run in the current working directory. If any or all of the above files +are not in that directory then you will not be able to run repy files. + + + + +### Tutorials for Repy and Python + +Now that you have Repy and Python, you may need a refresher on how to use them. +The following tutorials provide this information. + + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + + +## Building the security layer + +The following program is a sample security layer, it is not complete and does +not handle all cases required by the API. Remember, you have no idea how the +attacker will try to penetrate your security layer, so it is important that you +leave nothing to chance! + + +### A basic (and inadequate) defense + +Time to start coding! Let's inspect a basic security layer. +You can use the code given below as a template for your reference monitor. + +You may save the file as `reference_monitor_[netid].r2py`. + +```py +""" +This security layer inadequately handles the default functionality + +Note: + This security layer uses encasementlib.r2py, restrictions.default, repy.py and Python + Also you need to give it an application to run. + python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py + +""" +TYPE = "type" +ARGS = "args" +RETURN = "return" +EXCP = "exceptions" +TARGET = "target" +FUNC = "func" +OBJC = "objc" + + +class LPFile(): + def __init__(self, filename, create): + # globals + mycontext['debug'] = False + self.LPfile = openfile(filename, create) + + def readat(self, num_bytes, offset): + # Read from the file using the sandbox's readat... + return self.LPfile.readat(num_bytes, offset) + + def writeat(self, data, offset): + self.LPfile.writeat(data, offset) + + def close(self): + self.LPfile.close() + +def LPopenfile(filename, create): + return LPFile(filename, create) + +def LPremovefile(filename): + removefile(filename) + +# The code below sets up type checking and variable hiding for you. +# You should not change anything below this point. +sec_file_def = { + "obj-type": LPFile, + "name": "LPFile", + "writeat": {"type": "func", "args": (str, (int, long)), "exceptions": Exception, "return": (int, type(None)), "target": LPFile.writeat}, + "readat": {"type": "func", "args": ((int, long, type(None)), (int, long)), "exceptions": Exception, "return": str, "target": LPFile.readat}, + "close": {"type": "func", "args": None, "exceptions": Exception, "return": (bool, type(None)), "target": LPFile.close} +} + +CHILD_CONTEXT_DEF["openfile"] = { + TYPE: OBJC, + ARGS: (str, bool), + EXCP: Exception, + RETURN: sec_file_def, + TARGET: LPopenfile +} + +CHILD_CONTEXT_DEF["removefile"] = { + TYPE: FUNC, + ARGS: (str,), + EXCP: Exception, + RETURN: type(None), + TARGET: LPremovefile +} + +# Execute the user code +secure_dispatch_module() +``` + + +### Testing your security layer + +In this part of the assignment you will pretend to be an attacker. Remember the +attacker's objective is to bypass the restrictions or cause the security +layer to act in a disallowed manner. By understanding how the attacker thinks, +you will be able to write better security layers. + +A valid attack case is found below. + +You may save the file as `[netid]_attackcase.r2py`. + +```py +# Clean up if the files exist +if "default" in listfiles(): + removefile("default") +if "testfile.txt" in listfiles(): + removefile("testfile.txt") + +# Create a default file +default = openfile("default", True) + +# Initial write to default +default.writeat("TEMPLATE", 0) + +# Close default +default.close() + +# Open a file that doesn't exist +myfile = openfile("testfile.txt", False) + +# Read from the file. +# Passing None as first argument indicates that we want to read the whole file from offset 0. +assert myfile.readat(None, 0) == "TEMPLATE" + +# Close the file +myfile.close() +``` + +The sample defense layer will throw a `FileNotFoundError` if used as it is. But once it is implemented correctly, there should be no errors when testing with this attack case. + +**Note:** All attacks should be written as Repy V2 files, using the `.r2py` extension. + + +### Choice of File Names + +Filenames may only be in the current directory and may only contain lowercase +letters, numbers, the hyphen, underscore, and period characters. Also, filenames +cannot be '.', '..', the blank string or start with a period. There is no +concept of a directory or a folder in repy. Filenames must be no more than 120 +characters long. + + +### Running your security layer + +Finally, type the following command at the terminal to run your security layer +with your attack program. + +```bash +python repy.py restrictions.default encasementlib.r2py reference_monitor_[netid].r2py [netid]_attackcase.r2py +``` + +Make sure you went through the "How to get RepyV2" section! + + +# Notes and Resources + +* For a complete list of syntax in Repyv2 please visit: + **[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md]** + +* The following link is an excellent source for information about security + layers: + **[https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf]** + +* **Note:** It is possible to add multiple security layers to Repy, this may be +useful for testing different mitigations separately. This is done with the +following command at the terminal: + +```bash +python repy.py restrictions.default encasementlib.r2py [security_layer1].r2py [security_layer2].r2py [security_layer3].r2py [program].r2py +``` + + * **Your security layer must produce no output!!** + + * In repy log replaces print from python. This may be helpful when testing if +Repy installed correctly. + + +# What to turn in ? + +* Turn in a single repy file called `reference_monitor_[netid].r2py` with all +letters in lowercase. If your net id is abcd123, then the file you upload must be named `reference_monitor_abcd123.r2py`. + +* Your task is to implement the reference monitor to try to account for all scenarios implied by the given [Specifications](#specifications). + +* **Never raise unexpected errors or produce any output.** Your reference monitor must +produce no output when run normally, with a valid attack case. Make sure that you remove any `log()` statements used for debugging before submission. + From 45d46bef8ab67310bad1544f8c39bc4be0d54964 Mon Sep 17 00:00:00 2001 From: Rajath Reghunath Date: Tue, 17 Sep 2024 20:18:48 +0530 Subject: [PATCH 110/122] Apply suggestions from code review Co-authored-by: Justin Cappos --- EducationalAssignments/DefaultPartOne.md | 15 ++------------- 1 file changed, 2 insertions(+), 13 deletions(-) diff --git a/EducationalAssignments/DefaultPartOne.md b/EducationalAssignments/DefaultPartOne.md index a80b3a36..008fcf54 100644 --- a/EducationalAssignments/DefaultPartOne.md +++ b/EducationalAssignments/DefaultPartOne.md @@ -28,10 +28,10 @@ properly in different cases and to test attacks against your monitor. 1. Your defense monitor should incorporate all the standard file operation methods, from opening a file, reading and writing to it, to closing and deleting it. -2. In addition, a specially named file - `default` - shall be used as a template when opening files without creating them first. +2. In addition, if a specially named file - `default` - exists, it shall be used as a template when opening files without creating them first. Eg: Calling `openfile('foo', True)` should create and open a new empty file called `foo` (assuming it's not present already). However, calling `openfile('foo', False)` should create a new file using `default` as the template. If `default` doesn't exist, throw the relevant error (`FileNotFoundError`). -3. If default is created, written to or deleted, then all closed files that were previously created gets deleted. Any files that are already open must be left unchanged. +3. If default is created, written to, or deleted, then all closed files that were previously created gets deleted. Any files that are already open must be left unchanged. Three design paradigms are at work in this assignment: accuracy, efficiency, and security. @@ -93,17 +93,6 @@ restrictions.default, encasementlib.r2py, the security layer and the program you want to run in the current working directory. If any or all of the above files are not in that directory then you will not be able to run repy files. - ### Tutorials for Repy and Python From a82c711808c5e7365222b843b8dc050c83cbde4f Mon Sep 17 00:00:00 2001 From: Rajath Reghunath Date: Tue, 17 Sep 2024 13:49:17 -0400 Subject: [PATCH 111/122] Improve given sample code --- EducationalAssignments/DefaultPartOne.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/EducationalAssignments/DefaultPartOne.md b/EducationalAssignments/DefaultPartOne.md index 008fcf54..4ef96628 100644 --- a/EducationalAssignments/DefaultPartOne.md +++ b/EducationalAssignments/DefaultPartOne.md @@ -70,7 +70,7 @@ security layers and application that you want to run.) In order to test whether or not these steps worked, please copy and paste the code found below for the sample security layer and sample attack. -You should get a `FileNotFoundError` when running the test. If not, please go +You should not get any errors (or outputs) when running the test. If not, please go through the troubleshooting section found below. @@ -143,10 +143,17 @@ class LPFile(): def __init__(self, filename, create): # globals mycontext['debug'] = False - self.LPfile = openfile(filename, create) + + if create == False and 'default' in listfiles(): + default_file = openfile('default', False) + content = default_file.readat(None, 0) # Read from the file using the sandbox's readat + self.LPfile = openfile(filename, True) + self.LPfile.writeat(content, 0) + default_file.close() + else: + self.LPfile = openfile(filename, create) def readat(self, num_bytes, offset): - # Read from the file using the sandbox's readat... return self.LPfile.readat(num_bytes, offset) def writeat(self, data, offset): @@ -161,6 +168,7 @@ def LPopenfile(filename, create): def LPremovefile(filename): removefile(filename) + # The code below sets up type checking and variable hiding for you. # You should not change anything below this point. sec_file_def = { @@ -230,7 +238,8 @@ assert myfile.readat(None, 0) == "TEMPLATE" myfile.close() ``` -The sample defense layer will throw a `FileNotFoundError` if used as it is. But once it is implemented correctly, there should be no errors when testing with this attack case. +The given defense layer should not output any errors when used with this simple attack case. +However, you need to modify the reference monitor and the attack case to account for any possible scenario. **Note:** All attacks should be written as Repy V2 files, using the `.r2py` extension. From 8277ceb95d0902ca06b292d5648cdaa59cf5d259 Mon Sep 17 00:00:00 2001 From: Rajath Reghunath Date: Sat, 5 Oct 2024 20:53:15 -0400 Subject: [PATCH 112/122] Add DefaultPartTwo.md --- EducationalAssignments/DefaultPartTwo.md | 200 +++++++++++++++++++++++ 1 file changed, 200 insertions(+) create mode 100644 EducationalAssignments/DefaultPartTwo.md diff --git a/EducationalAssignments/DefaultPartTwo.md b/EducationalAssignments/DefaultPartTwo.md new file mode 100644 index 00000000..da13c89f --- /dev/null +++ b/EducationalAssignments/DefaultPartTwo.md @@ -0,0 +1,200 @@ +# Security layer testing and penetration + +In this assignment you will learn how to attack a reference monitor. The +reference monitor you will be testing uses the security layer framework +(encasement library, etc.) for the Seattle testbed. It is possible to do this +assignment separately, but it is recommended that this assignment be completed +after [Part One](./DefaultPartOne.md). Either way you should already have a working +security layer or access to one. Testing the security layer is done by running a +series of test cases that an adversary may use to circumvent your system. This +assignment is intended to prepare you for thinking about security paradigms in a +functional way. The ideas of information, security and privacy have been +embedded into the steps of this assignment. + + +## Overview + +In this assignment you are a tester. You have been sent a bunch of reference +monitors that need testing before they are deployed. Your job will be to ensure +an attacker cannot circumvent these security layers. In order to do this, you +will attempt to write, and append invalid data. If you are able to do so, then +the security layer is not secure. The future of the system depends on your +ability to test code thoroughly! + +Three design paradigms are at work in this assignment: accuracy, efficiency, and +security. + + * Accuracy: The security layer should only stop certain actions from being + blocked. All other actions should be allowed. + + * Efficiency: The security layer should use a minimum number of resources, so + performance is not compromised. + + * Security: The attacker should not be able to circumvent the security layer. + + +Within the context of this assignment these design paradigms translate to: + + * Accuracy: The defense monitor should precisely and consistently manage file +operations. Only specific operations, such as `writeat`, should have their +behavior modified. All situations that are not described in the specifications +*must* match that of the underlying API. + + * Efficiency: The security layer should use a minimum number of resources, so +performance is not compromised. For example, you may not call `readat()` +everytime `writeat()` is called. It is permissable to call `readat()` upon +`fileopen()`, however. + + * Security: The defense layer should be robust against tampering and +circumvention. Attackers must not be able to bypass, disable, or exploit the +defense monitor's enhanced behaviors, ensuring the integrity and intended +functionality of file operations. + +You will submit a zip file containing all of the tests you have created. You +will gain points for every student's reference monitor you find a flaw in. It is +good if multiple tests of yours break a student's reference monitor, but you +gain the same number of tests whether one or more tests break the layer. + + +## Prerequisites + +This assignment assumes you have both the latest Python 2.7 and RepyV2 installed +on your computer. Please refer to the [SeattleTestbed Build +Instructions](../Contributing/BuildInstructions.md#prerequisites) for +information on how to get them. + + +### Helpful links + +The following links will aid students in becoming comfortable with Python, Repy +and seattle: + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + + +## Testing security layers + +### Hypothesis, test case, counter example + +The goal of a good tester is to test hypotheses. A hypothesis is just a +scientific way of asking a question. The hypothesis of this assignment is "This +security layer is well designed." The questions you will ask when running your +test cases will always be the same + + * "Is this reference monitor secure?" + + * "Does this reference monitor hamper performance?" + + * "Does this reference monitor prevent actions that should be allowed?" + +Notice that these questions are parallels of the security paradigms: security, +efficiency and accuracy, respectively. + +If we can find a case where the hypothesis is false, then the security layer is +not secure. Such a case is referred to as a counter example. Hence all test +cases should be designed to test for these three types of flaws. + +#### Information on: Try, Except, Else, Finally + +The try, except, else and finally statements are part of **exception handling**. +For more information on exception handling please visit: + + * [http://docs.python.org/tutorial/errors.html] + * [http://wiki.python.org/moin/HandlingExceptions] + * [http://www.tutorialspoint.com/python/python_exceptions.htm] + +### Hints and Ideas for testing + +When writing your own tests it is important to test for a complete set of +possible penetrations. Keep in mind, it only takes one test case to break +through a security layer. Some of the things you may want to test for include: + + * threading + * multiple writes + +And more! Remember a good security layer can't be broken by anyone! Which is +all part of the fun! It's about solving a puzzle. First you make the puzzle - +write the security layer, then you solve the puzzle - try to bypass it. If your +puzzle is "good enough", no one will be able to break it, no matter what. + + +> Note: When creating tests, the best practice is to separate them into +> different files. Each test file should focus on testing one specific scenario +> or functionality. If there is a flaw in any part of a test file, we will +> discard the entire test file. This modular approach helps isolate issues - if +> there is a flaw in one test file, it will not affect or invalidate the other +> test files, since each file is independent and testing distinct functionality. + + +## Notes and Resources + + * The following link is an excellent source for information about security + layers: https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf + + * In repy `log` replaces `print` from python. Many students find this to be a + stumbling block. + + * Note that you should not assume that any files exist in your directory. You + should create any files (e.g., `testfile.txt`) yourself in your test program. + + * It's important to note that if a test has a flaw in any part of it, the + entire test will be considered invalid. So, it's advisable to break your + tests into different files. + +## How to run your tests on many reference monitors + +Create a directory that the security layers will write their files into. You +need to run repy with only access to this directory. You can write a test +program that does `log(str(listfiles()))` to see if you are in the right place. + +Have all the reference monitors to test and the test cases inside the same +directory where the `repy.py` file exists. + +* In the bash shell on Mac and Linux: +```bash +for referencemonitor in reference_monitor_*; do for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` + +* In the Command Prompt on Windows: +```cmd +FOR %r IN (reference_monitor_*) DO @FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a +``` + +* In PowerShell on Windows: +```powershell +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the output from each program. Make sure that you replace +`` with your NetID. + +If you want to spot the reference monitor that failed during the test run, add +echo the name of each reference monitor before the inner loop, like so: + +* In the bash shell on Mac and Linux: +```bash +for referencemonitor in reference_monitor_*; do echo $referencemonitor under test; for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` + +* In the Command Prompt on Windows: +```cmd +FOR %r IN (reference_monitor_*) DO @(ECHO %r under test & FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a) +``` + +* In PowerShell on Windows: +```powershell +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { Write-Host $referencemonitor.Name; foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the name of each reference monitor before it starts +executing the test cases against it. + + +## What to turn in? + + * Never raise unexpected errors or produce any output. Your attack must produce + no output when run normally. + * Turn in the test cases used to attack a given reference monitor in a zip + file. The name of each testcase must match the following format: `[ net_id + ]_attackcase1.r2py`, `[ net_id ]_attackcase2.r2py`, etc. From c55766d7a4e3bfbfa758b6a8418f8177bdf13e89 Mon Sep 17 00:00:00 2001 From: Rajath Reghunath Date: Sat, 5 Oct 2024 21:29:14 -0400 Subject: [PATCH 113/122] edits --- EducationalAssignments/DefaultPartTwo.md | 28 +++++++++++++----------- 1 file changed, 15 insertions(+), 13 deletions(-) diff --git a/EducationalAssignments/DefaultPartTwo.md b/EducationalAssignments/DefaultPartTwo.md index da13c89f..18e8a0e8 100644 --- a/EducationalAssignments/DefaultPartTwo.md +++ b/EducationalAssignments/DefaultPartTwo.md @@ -17,8 +17,10 @@ embedded into the steps of this assignment. In this assignment you are a tester. You have been sent a bunch of reference monitors that need testing before they are deployed. Your job will be to ensure an attacker cannot circumvent these security layers. In order to do this, you -will attempt to write, and append invalid data. If you are able to do so, then -the security layer is not secure. The future of the system depends on your +will attempt to write testcases that check if the reference monitors behave as +they should for any valid action that can be taken by a user. These testcases +will try to trigger unexpected behaviours in the reference monitor. If they are +able to do so, then the security layer is not secure. The future of the system depends on your ability to test code thoroughly! Three design paradigms are at work in this assignment: accuracy, efficiency, and @@ -36,14 +38,13 @@ security. Within the context of this assignment these design paradigms translate to: * Accuracy: The defense monitor should precisely and consistently manage file -operations. Only specific operations, such as `writeat`, should have their +operations. Only specific operations, such as `openfile`, should have their behavior modified. All situations that are not described in the specifications *must* match that of the underlying API. * Efficiency: The security layer should use a minimum number of resources, so -performance is not compromised. For example, you may not call `readat()` -everytime `writeat()` is called. It is permissable to call `readat()` upon -`fileopen()`, however. +performance is not compromised. For example, it is not permissible to store the +contents of `default` in memory all the time, except when it's being copied. * Security: The defense layer should be robust against tampering and circumvention. Attackers must not be able to bypass, disable, or exploit the @@ -53,7 +54,7 @@ functionality of file operations. You will submit a zip file containing all of the tests you have created. You will gain points for every student's reference monitor you find a flaw in. It is good if multiple tests of yours break a student's reference monitor, but you -gain the same number of tests whether one or more tests break the layer. +gain the same number of points whether one or more tests break the layer. ## Prerequisites @@ -111,7 +112,8 @@ possible penetrations. Keep in mind, it only takes one test case to break through a security layer. Some of the things you may want to test for include: * threading - * multiple writes + * correct error handling + * files being deleted correctly And more! Remember a good security layer can't be broken by anyone! Which is all part of the fun! It's about solving a puzzle. First you make the puzzle - @@ -193,8 +195,8 @@ executing the test cases against it. ## What to turn in? - * Never raise unexpected errors or produce any output. Your attack must produce - no output when run normally. - * Turn in the test cases used to attack a given reference monitor in a zip - file. The name of each testcase must match the following format: `[ net_id - ]_attackcase1.r2py`, `[ net_id ]_attackcase2.r2py`, etc. +* Turn in all the test cases in a zip file. The name of each testcase must match the following format: `_attackcase.r2py`. For example, if your netid id `abc123`, then the submitted zip files must contain a set of files named as `abc123_attackcase1.r2py`, `abc123_attackcase2.r2py`. There are no restrictions on the number of attackcases that can be submitted. + +* **Never raise unexpected errors or produce any output.** Your attackcase must not produce any error or output, if the reference monitor is behaving as it should. This applies for error checking as well. If the attackcase tries to trigger an expected error, it should catch and suppress that error if the reference monitor correctly raises that error. + +* While you're allowed to test if the correct errors are being raised, you aren't allowed to check the error message for any errors. That means checking for `FileInUseError` is valid, but it's not valid to check the specific error message. \ No newline at end of file From 9ef95bb4a78d518281959097a5f6737c871db33f Mon Sep 17 00:00:00 2001 From: Rajath Reghunath Date: Sun, 6 Oct 2024 13:21:49 -0400 Subject: [PATCH 114/122] edits 2 --- EducationalAssignments/DefaultPartTwo.md | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/EducationalAssignments/DefaultPartTwo.md b/EducationalAssignments/DefaultPartTwo.md index 18e8a0e8..9e302b29 100644 --- a/EducationalAssignments/DefaultPartTwo.md +++ b/EducationalAssignments/DefaultPartTwo.md @@ -44,7 +44,8 @@ behavior modified. All situations that are not described in the specifications * Efficiency: The security layer should use a minimum number of resources, so performance is not compromised. For example, it is not permissible to store the -contents of `default` in memory all the time, except when it's being copied. +contents of `default` in memory all the time. However, this is allowed when you're +copying the contents of default to a new file. * Security: The defense layer should be robust against tampering and circumvention. Attackers must not be able to bypass, disable, or exploit the @@ -199,4 +200,6 @@ executing the test cases against it. * **Never raise unexpected errors or produce any output.** Your attackcase must not produce any error or output, if the reference monitor is behaving as it should. This applies for error checking as well. If the attackcase tries to trigger an expected error, it should catch and suppress that error if the reference monitor correctly raises that error. -* While you're allowed to test if the correct errors are being raised, you aren't allowed to check the error message for any errors. That means checking for `FileInUseError` is valid, but it's not valid to check the specific error message. \ No newline at end of file +* While you're allowed to test if the correct errors are being raised, you aren't allowed to check the error message for any errors. That means checking for `FileInUseError` is valid, but it's not valid to check the specific error message. + +* Attackcases should be independent of each other. This means that you should not try to use any file (or it's contents) across different attackcases. \ No newline at end of file From 19cf61e8d62459ddcc898c5da16bcb798a1c9f0f Mon Sep 17 00:00:00 2001 From: Rajath Reghunath Date: Tue, 8 Oct 2024 00:25:55 -0400 Subject: [PATCH 115/122] minor edit to DefaultPartOne --- EducationalAssignments/DefaultPartOne.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/EducationalAssignments/DefaultPartOne.md b/EducationalAssignments/DefaultPartOne.md index 4ef96628..61bed479 100644 --- a/EducationalAssignments/DefaultPartOne.md +++ b/EducationalAssignments/DefaultPartOne.md @@ -42,9 +42,9 @@ behavior modified. All situations that are not described above *must* match that of the underlying API. * Efficiency: The security layer should use a minimum number of resources, so -performance is not compromised. For example, you may not call `readat()` -everytime `writeat()` is called. It is permissable to call `readat()` upon -`fileopen()`, however. +performance is not compromised. For example, it is not permissible to store the +contents of `default` in memory all the time. However, this is allowed when you're +copying the contents of default to a new file. * Security: The defense layer should be robust against tampering and circumvention. Attackers must not be able to bypass, disable, or exploit the From cfa2464ce2c14e8bd2cb0158e696e3cb2949b885 Mon Sep 17 00:00:00 2001 From: Rajath Reghunath Date: Mon, 21 Oct 2024 12:55:34 -0400 Subject: [PATCH 116/122] Add draft for ISP Fall 2024 Assignment 2.3 --- EducationalAssignments/DefaultPartThree.md | 88 ++++++++++++++++++++++ 1 file changed, 88 insertions(+) create mode 100644 EducationalAssignments/DefaultPartThree.md diff --git a/EducationalAssignments/DefaultPartThree.md b/EducationalAssignments/DefaultPartThree.md new file mode 100644 index 00000000..e2324608 --- /dev/null +++ b/EducationalAssignments/DefaultPartThree.md @@ -0,0 +1,88 @@ +# Fixing your security layer + +In this assignment you will analyze the bugs in your security layer from [Part +One](./DefaultPartOne.md) +and fix them. You may want to use test cases from [Part +Two](./DefaultPartTwo.md) +to help identify these bugs. + + +## Overview + +In this assignment you are fixing your reference monitor. You have been given a +bunch of test programs that try to compromise your reference monitor. Your job +will be to determine where your reference monitor failed, and fix it to ensure +an attacker cannot circumvent the security layer. + +## Code Analysis + +Given below is a section taken from the sample reference monitor given +in [Part One](./DefaultPartOne.md#a-basic-and-inadequate-defense). + +```py +class LPFile(): + def __init__(self, filename, create): + # globals + mycontext['debug'] = False + + if create == False and 'default' in listfiles(): + # bug ? + default_file = openfile('default', False) + content = default_file.readat(None, 0) + self.LPfile = openfile(filename, True) + self.LPfile.writeat(content, 0) + default_file.close() + else: + # bug ? + self.LPfile = openfile(filename, create) + + def readat(self, num_bytes, offset): + return self.LPfile.readat(num_bytes, offset) + + def writeat(self, data, offset): + # bug ? + self.LPfile.writeat(data, offset) + + def close(self): + # bug ? + self.LPfile.close() + +def LPopenfile(filename, create): + return LPFile(filename, create) + +def LPremovefile(filename): + # bug ? + removefile(filename) +``` + +Let's analyze some of the bugs in this implementation (not in any specific order). + +1. As outlined in the specifications in [PartOne](./DefaultPartOne.md), if the `default` file is absent, +and we try to open a non-existent file with create=False, then the relevant +error (`FileNotFoundError`) must be thrown. This scenario is not correctly handled in the sample implementation. + +2. Similarly, if the `default` file is modified (created/written to/deleted), then we need to delete all files that were created +during the run and are not open at the moment of modification. To achieve this, the state of all (open) files created during the +run should be tracked. You can use the global variable `mycontext` to store this state. For efficiency, only filenames or +file handles should be stored, rather than the actual file contents. + +3. One common edge case often overlooked by students occurs when attempting to open a non-existent file with +create=False while default is already open. If it's not handled correctly, this will cause a FileInUseError, which +shouldn't happen. To prevent this, the reference monitor should maintain a reference to the file handle of default, +whenever default is open (so that it can be used to create new templates). Also note that storing the contents of default +in memory at all times is not an optimal solution due to efficiency concerns. + +4. Finally, this example doesn't use locks, which can lead to race conditions, thus allowing various types of multi-threading attacks. + + +## What to turn in? + +* Submit 2 files: + 1. 1 r2py file: Your reference monitor should be named as `reference_monitor_[netid].r2py` with all +letters in lowercase . For example, if your netid is `abc123`, the file should be named: `reference_monitor_abc123.r2py`. + 2. 1 pdf file: Write up a 1 page report explaining the rationale behind the vulnerabilities and how you fixed them. Name it `[netid].pdf`. + +* **Never raise unexpected errors or produce any output.** Your reference monitor must +produce no output when run normally, with a valid attack case. Make sure that you remove any `log()` statements used for debugging before submission. + +* You are required to modify your reference monitor so that it generates no output (logs or errors) when any valid attack case (from Part 2.2, after invalid attack cases have been identified and excluded) is executed against it. From 019c113340784e772bf79f1eb07d11b252752c82 Mon Sep 17 00:00:00 2001 From: Harini Vinu Date: Sun, 5 Oct 2025 20:52:48 -0400 Subject: [PATCH 117/122] Adding Assignment 2.1 2025 Immutable-versioned files security layer --- EducationalAssignments/ImmutableVersionOne.md | 262 ++++++++++++++++++ 1 file changed, 262 insertions(+) create mode 100644 EducationalAssignments/ImmutableVersionOne.md diff --git a/EducationalAssignments/ImmutableVersionOne.md b/EducationalAssignments/ImmutableVersionOne.md new file mode 100644 index 00000000..8225e49a --- /dev/null +++ b/EducationalAssignments/ImmutableVersionOne.md @@ -0,0 +1,262 @@ +# Implement a Defensive Security System + + +This assignment will help you understand security mechanisms. You will be guided through the steps of creating a reference monitor using the security layer functionality in Repy V2. A reference monitor is an access control concept that refers to an abstract machine that mediates all access to objects by subjects. This can be used to allow, deny, or change the behavior of any set of calls. + + +One critical aspect of creating a reference monitor is to ensure it cannot be bypassed and handles all cases correctly. While not a perfect way of validating your reference monitor, it is useful to create test cases to see whether your security layer will work as expected (the test cases may be turned in as part of the next assignment). + + +This assignment is intended to reinforce concepts of immutability, access control, and state consistency. By the end, you should understand how to design a security layer that preserves history and enforces tamper resistance. + +## Overview + +In this assignment, you will implement a defense monitor to enforce immutable, versioned files in Repy. The monitor will enhance the default Repy file behavior by ensuring that once a file version is closed, it can never be modified again, and new versions are created in a controlled, linear order. +You should write test applications to ensure your reference monitor behaves properly in different cases and to test attacks against your monitor. + +## Specifications + +1. Your defense monitor should incorporate all the standard file operation methods, from opening a file, reading and writing to it, to closing it. All operations must behave identically to RepyV2 (without your security layer) except as mentioned below. +2. You can assume that no files exist when your security layer begins running the application. +3. When a user calls `openfile(filename, True)`, if `filename` already exists, your security layer must create a new “version” of the file that can be opened. This version will be given a new version number and must begin with the contents of the latest version file as its contents. +4. Note that a new version cannot be created while the latest version is open. If `openfile()` is called on an already open file, it shall throw the relevant error that it would have done in RepyV2 (`FileInUseError`). +5. Versioned files can be accessed using `openfile(originalfilename + '.v' + str(num), create)`, where `num` starts from 1. If `create=True` is used on a versioned file, raise `RepyArgumentError("Cannot create explicit version files")`, as manual version creation is not allowed. +6. As is the case in normal RepyV2, if an `openfile` call with `create=False`, open the file only if that version exists; otherwise, raise `FileNotFoundError`. +7. Reading from older versions is always allowed, but writing to them is permanently disallowed. Any attempt to write to an older version should raise a `FileInUseError`. +8. File deletion (`removefile`) is not allowed for any file. Any attempt must raise a `RepyArgumentError`. +9. The `listfiles()` call in RepyV2 should not show that versions of files exist. It should just show the listing of all files that were created, regardless of how many versions of those file exist. + +## Three Design Paradigms + +As in earlier assignments, three paradigms are at work: + +- **Accuracy:** The defense monitor should precisely and consistently manage file operations. All unspecified actions must behave exactly as in the underlying API. +- **Efficiency:** The security layer should use minimal resources. Do not cache or copy entire files in memory—only copy when creating a new version. +- **Security:** The defense layer should be robust against tampering and circumvention. Attackers must not be able to bypass, disable, or exploit the defense monitor’s enhanced behaviors, ensuring that immutability and versioning are always enforced as intended. + +## Getting Python and RepyV2 + +Please refer to the [SeattleTestbed Build +Instructions](../Contributing/BuildInstructions.md#prerequisites) for details. + +Once you have built RepyV2 into a directory of your choice, change into that directory. Use the command below in order to run your RepyV2 applications: + +```bash +python2 repy.py restrictions.default encasementlib.r2py [security_layer].r2py [application].r2py +``` + +(Replace `[security_layer].r2py` and `[application].r2py` by the names of the security layers and application that you want to run.) + +In order to test whether or not these steps worked, please copy and paste the code found below for the sample security layer and sample attack. + +You should not get any errors (or outputs) when running the test. If not, please go through the troubleshooting section found below. + +### Troubleshooting Repy code +--- + +If you can't get Repy files to run, some of the following common errors may have occurred: + +- **using `print` instead of `log`:** + Repy is a subset of Python, but its syntax is slightly different. For example, Python's `print` statement cannot be used; Repy has `log` for that. For a full list of acceptable syntax please see + + +- **command line errors:** + + **files are missing:** In the above command line call, you must have `repy.py`, `restrictions.default`, `encasementlib.r2py`, the security layer and the program you want to run in the current working directory. If any or all of the above files are not in that directory then you will not be able to run repy files. + +### Tutorials for Repy and Python +---- + +Now that you have Repy and Python, you may need a refresher on how to use them. +The following tutorials provide this information. + + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + +## Building the security layer +------ + +The following program is a sample security layer, it is not complete and does not handle all cases required by the API. Remember, you have no idea how the attacker will try to penetrate your security layer, so it is important that you leave nothing to chance! + +#### A basic (and inadequate) defense + +Time to start coding! Let's inspect a basic security layer. You can use the code given below as a template for your reference monitor. +You may save the file as `reference_monitor_[netid].r2py`. + +```py +""" +This security layer inadequately handles the Versioned and Immutable functionality + +Note: + This security layer uses encasementlib.r2py, restrictions.default, repy.py and Python + Also you need to give it an application to run. + python repy.py restrictions.default encasementlib.r2py [security_layer].r2py [attack_program].r2py + +""" +TYPE = "type" +ARGS = "args" +RETURN = "return" +EXCP = "exceptions" +TARGET = "target" +FUNC = "func" +OBJC = "objc" + + +class VMFile(): + def __init__(self, filename, create): + if create: + self.VMfile = openfile(filename, True) + else: + self.VMfile = openfile(filename, False) + + def readat(self, num_bytes, offset): + return self.VMfile.readat(num_bytes, offset) + + def writeat(self, data, offset): + return self.VMfile.writeat(data, offset) + + def close(self): + return self.VMfile.close() + + +def LPopenfile(filename, create): + return VMFile(filename, create) + +def LPremovefile(filename): + removefile(filename) + +def LPlistfiles(): + return listfiles() + + +# The code below sets up type checking and variable hiding for you. +# You should not change anything below this point. +sec_file_def = { + "obj-type": VMFile, + "name": "VMFile", + "writeat": {"type": "func", "args": (str, (int, long)), "exceptions": Exception, "return": (int, type(None)), "target": VMFile.writeat}, + "readat": {"type": "func", "args": ((int, long, type(None)), (int, long)), "exceptions": Exception, "return": str, "target": VMFile.readat}, + "close": {"type": "func", "args": None, "exceptions": Exception, "return": (bool, type(None)), "target": VMFile.close} +} + +CHILD_CONTEXT_DEF["openfile"] = { + TYPE: OBJC, + ARGS: (str, bool), + EXCP: Exception, + RETURN: sec_file_def, + TARGET: LPopenfile +} + +CHILD_CONTEXT_DEF["removefile"] = { + TYPE: FUNC, + ARGS: (str,), + EXCP: Exception, + RETURN: type(None), + TARGET: LPremovefile +} + +CHILD_CONTEXT_DEF["listfiles"] = { + TYPE: FUNC, + ARGS: None, + EXCP: Exception, + RETURN: [str], + TARGET: LPlistfiles +} + +# Execute the user code +secure_dispatch_module() +``` + + +### Testing your security layer + +In this part of the assignment you will pretend to be an attacker. Remember the +attacker's objective is to bypass the restrictions or cause the security +layer to act in a disallowed manner. By understanding how the attacker thinks, +you will be able to write better security layers. + +A valid attack case is found below. + +You may save the file as `[netid]_attackcase.r2py`. + +```py +# --- Create first version of "testfile" --- +# Calling openfile with create=True should create a new version. +f1 = openfile("testfile", True) + +# Write initial content at offset 0. +f1.writeat("HelloWorld", 0) + +# Close the file to finalize the first version's contents. +f1.close() + +# --- Create second version of "testfile" --- +# Calling openfile again with create=True should create the next version, and copy the previous version's contents into the new version. +f2 = openfile("testfile", True) + +# Read the entire file from offset 0 (pass None to read the whole file) and verify the copy occurred. +assert f2.readat(None, 0) == "HelloWorld" + +# Close v2. +f2.close() +``` +The given defense layer should not output any errors when used with this simple attack case. +However, you need to modify the reference monitor and the attack case to account for any possible scenario. + +**Note:** All attacks should be written as Repy V2 files, using the `.r2py` extension. + + +### Choice of File Names + +Filenames may only be in the current directory and may only contain lowercase +letters, numbers, the hyphen, underscore, and period characters. Also, filenames +cannot be '.', '..', the blank string or start with a period. There is no +concept of a directory or a folder in repy. Filenames must be no more than 120 +characters long. + + +### Running your security layer + +Finally, type the following command at the terminal to run your security layer +with your attack program. + +```bash +python repy.py restrictions.default encasementlib.r2py reference_monitor_[netid].r2py [netid]_attackcase.r2py +``` + +Make sure you went through the "How to get RepyV2" section! + + +# Notes and Resources + +* For a complete list of syntax in Repyv2 please visit: + **[https://github.com/SeattleTestbed/docs/blob/master/Programming/RepyV2API.md]** + +* The following link is an excellent source for information about security + layers: + **[https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf]** + +* **Note:** It is possible to add multiple security layers to Repy, this may be +useful for testing different mitigations separately. This is done with the +following command at the terminal: + +```bash +python repy.py restrictions.default encasementlib.r2py [security_layer1].r2py [security_layer2].r2py [security_layer3].r2py [program].r2py +``` + + * **Your security layer must produce no output!!** + + * In repy log replaces print from python. This may be helpful when testing if +Repy installed correctly. + + +# What to turn in ? + +* Turn in a single repy file called `reference_monitor_[netid].r2py` with all +letters in lowercase. If your net id is abcd123, then the file you upload must be named `reference_monitor_abcd123.r2py`. + +* Your task is to implement the reference monitor to try to account for all scenarios implied by the given specifications. + +* **Never raise unexpected errors or produce any output.** Your reference monitor must +produce no output when run normally, with a valid attack case. Make sure that you remove any `log()` statements used for debugging before submission. \ No newline at end of file From 3d4bd35b673a20ed8809972073b724b4f800a46b Mon Sep 17 00:00:00 2001 From: vgharini Date: Mon, 6 Oct 2025 18:40:31 -0400 Subject: [PATCH 118/122] incorporated comments --- EducationalAssignments/ImmutableVersionOne.md | 26 ++++++++++++++----- 1 file changed, 20 insertions(+), 6 deletions(-) diff --git a/EducationalAssignments/ImmutableVersionOne.md b/EducationalAssignments/ImmutableVersionOne.md index 8225e49a..33eb5c5e 100644 --- a/EducationalAssignments/ImmutableVersionOne.md +++ b/EducationalAssignments/ImmutableVersionOne.md @@ -7,7 +7,8 @@ This assignment will help you understand security mechanisms. You will be guided One critical aspect of creating a reference monitor is to ensure it cannot be bypassed and handles all cases correctly. While not a perfect way of validating your reference monitor, it is useful to create test cases to see whether your security layer will work as expected (the test cases may be turned in as part of the next assignment). -This assignment is intended to reinforce concepts of immutability, access control, and state consistency. By the end, you should understand how to design a security layer that preserves history and enforces tamper resistance. +This assignment is intended to reinforce concepts of immutability, access control, and state consistency, where immutability ensures that once data is written, it cannot be changed or deleted; instead, new versions are created to preserve a tamper-proof history for auditing and recovery. By the end, you should understand how to design a security layer that maintains this history and enforces tamper resistance. + ## Overview @@ -16,9 +17,9 @@ You should write test applications to ensure your reference monitor behaves prop ## Specifications -1. Your defense monitor should incorporate all the standard file operation methods, from opening a file, reading and writing to it, to closing it. All operations must behave identically to RepyV2 (without your security layer) except as mentioned below. +1. Your defense monitor should incorporate all the standard file operation methods supported in RepyV2, from opening a file, reading and writing to it,listing files, deleting files, and to closing them. All operations must behave identically to RepyV2 (without your security layer) except as mentioned below. 2. You can assume that no files exist when your security layer begins running the application. -3. When a user calls `openfile(filename, True)`, if `filename` already exists, your security layer must create a new “version” of the file that can be opened. This version will be given a new version number and must begin with the contents of the latest version file as its contents. +3. When a user calls `openfile(filename, True)`, if `filename` already exists, your security layer must create a new “version” — a new file initialized with the contents of the latest version and given a new version number. 4. Note that a new version cannot be created while the latest version is open. If `openfile()` is called on an already open file, it shall throw the relevant error that it would have done in RepyV2 (`FileInUseError`). 5. Versioned files can be accessed using `openfile(originalfilename + '.v' + str(num), create)`, where `num` starts from 1. If `create=True` is used on a versioned file, raise `RepyArgumentError("Cannot create explicit version files")`, as manual version creation is not allowed. 6. As is the case in normal RepyV2, if an `openfile` call with `create=False`, open the file only if that version exists; otherwise, raise `FileNotFoundError`. @@ -105,10 +106,23 @@ OBJC = "objc" class VMFile(): def __init__(self, filename, create): - if create: - self.VMfile = openfile(filename, True) + # If a file with the same 'filename' already exists, this creates a new version 'filename.v1'. + # (Incomplete: does not handle further versions like v2, v3, etc.) + if create: + if filename in listfiles(): + # File exists → create version 1 + prev_file = openfile(filename, False) + content = prev_file.readat(None, 0) + + new_name = filename + ".v1" + self.VMfile = openfile(new_name, True) + self.VMfile.writeat(content, 0) else: - self.VMfile = openfile(filename, False) + # File doesn't exist → create filename + self.VMfile = openfile(filename, True) + else: + # Open existing file normally + self.VMfile = openfile(filename, False) def readat(self, num_bytes, offset): return self.VMfile.readat(num_bytes, offset) From c38c16c9b63fce2101881a8dd718bd5c649b4a72 Mon Sep 17 00:00:00 2001 From: vgharini Date: Wed, 8 Oct 2025 16:21:18 -0400 Subject: [PATCH 119/122] Fixed indentation issue --- EducationalAssignments/ImmutableVersionOne.md | 28 +++++++++---------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/EducationalAssignments/ImmutableVersionOne.md b/EducationalAssignments/ImmutableVersionOne.md index 33eb5c5e..4c3ef685 100644 --- a/EducationalAssignments/ImmutableVersionOne.md +++ b/EducationalAssignments/ImmutableVersionOne.md @@ -108,21 +108,21 @@ class VMFile(): def __init__(self, filename, create): # If a file with the same 'filename' already exists, this creates a new version 'filename.v1'. # (Incomplete: does not handle further versions like v2, v3, etc.) - if create: - if filename in listfiles(): - # File exists → create version 1 - prev_file = openfile(filename, False) - content = prev_file.readat(None, 0) - - new_name = filename + ".v1" - self.VMfile = openfile(new_name, True) - self.VMfile.writeat(content, 0) + if create: + if filename in listfiles(): + # File exists → create version 1 + prev_file = openfile(filename, False) + content = prev_file.readat(None, 0) + + new_name = filename + ".v1" + self.VMfile = openfile(new_name, True) + self.VMfile.writeat(content, 0) + else: + # File doesn't exist → create filename + self.VMfile = openfile(filename, True) else: - # File doesn't exist → create filename - self.VMfile = openfile(filename, True) - else: - # Open existing file normally - self.VMfile = openfile(filename, False) + # Open existing file normally + self.VMfile = openfile(filename, False) def readat(self, num_bytes, offset): return self.VMfile.readat(num_bytes, offset) From cc5d7fd664c7f2ab0e9e398cd836adf2ae5df8b2 Mon Sep 17 00:00:00 2001 From: vgharini Date: Mon, 20 Oct 2025 13:47:33 -0400 Subject: [PATCH 120/122] listfiles --- EducationalAssignments/ImmutableVersionOne.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/EducationalAssignments/ImmutableVersionOne.md b/EducationalAssignments/ImmutableVersionOne.md index 4c3ef685..621f2e93 100644 --- a/EducationalAssignments/ImmutableVersionOne.md +++ b/EducationalAssignments/ImmutableVersionOne.md @@ -174,7 +174,7 @@ CHILD_CONTEXT_DEF["listfiles"] = { TYPE: FUNC, ARGS: None, EXCP: Exception, - RETURN: [str], + RETURN: list, TARGET: LPlistfiles } From 585d021c7376b874180569d329aa031bc4d58f96 Mon Sep 17 00:00:00 2001 From: vgharini Date: Tue, 21 Oct 2025 18:11:55 -0400 Subject: [PATCH 121/122] ImmutableVersionTwo --- EducationalAssignments/ImmutableVersionTwo.md | 193 ++++++++++++++++++ 1 file changed, 193 insertions(+) create mode 100644 EducationalAssignments/ImmutableVersionTwo.md diff --git a/EducationalAssignments/ImmutableVersionTwo.md b/EducationalAssignments/ImmutableVersionTwo.md new file mode 100644 index 00000000..c492a5fa --- /dev/null +++ b/EducationalAssignments/ImmutableVersionTwo.md @@ -0,0 +1,193 @@ +# Security layer testing and penetration + +In this assignment you will learn how to attack a reference monitor. The +reference monitor you will be testing uses the security layer framework +(encasement library, etc.) for the Seattle testbed. It is possible to do this +assignment separately, but it is recommended that this assignment be completed +after [Part One](./ImmutableVersionOne.md). Either way you should already have a working +security layer or access to one. Testing the security layer is done by running a +series of test cases that an adversary may use to circumvent your system. This +assignment is intended to prepare you for thinking about security paradigms in a +functional way. The ideas of information, security and privacy have been +embedded into the steps of this assignment. + + +## Overview + +In this assignment you are a tester. You have been sent a bunch of reference +monitors that need testing before they are deployed. Your job will be to ensure +an attacker cannot circumvent these security layers. In order to do this, you +will attempt to write testcases that check if the reference monitors behave as +they should for any valid action that can be taken by a user. These testcases +will try to trigger unexpected behaviours in the reference monitor. If they are +able to do so, then the security layer is not secure. The future of the system depends on your +ability to test code thoroughly! + +Three design paradigms are at work in this assignment: accuracy, efficiency, and +security. + + * Accuracy: The security layer should only stop certain actions from being + blocked. All other actions should be allowed. + + * Efficiency: The security layer should use a minimum number of resources, so + performance is not compromised. + + * Security: The attacker should not be able to circumvent the security layer. + + +Within the context of this assignment these design paradigms translate to: + + - **Accuracy:** The defense monitor should precisely and consistently manage file operations. All unspecified actions must behave exactly as in the underlying API. +- **Efficiency:** The security layer should use minimal resources. Do not cache or copy entire files in memory—only copy when creating a new version. +- **Security:** The defense layer should be robust against tampering and circumvention. Attackers must not be able to bypass, disable, or exploit the defense monitor’s enhanced behaviors, ensuring that immutability and versioning are always enforced as intended. + +You will submit a zip file containing all of the tests you have created. You +will gain points for every student's reference monitor you find a flaw in. It is +good if multiple tests of yours break a student's reference monitor, but you +gain the same number of points whether one or more tests break the layer. + + +## Prerequisites + +This assignment assumes you have both the latest Python 2.7 and RepyV2 installed +on your computer. Please refer to the [SeattleTestbed Build +Instructions](../Contributing/BuildInstructions.md#prerequisites) for +information on how to get them. + + +### Helpful links + +The following links will aid students in becoming comfortable with Python, Repy +and seattle: + * Official [Python tutorial](http://docs.python.org/tutorial/) + * [Differences between RepyV2 and Python](../Programming/PythonVsRepyV2.md) + * List of [RepyV2 API calls](../Programming/RepyV2API.md) + + +## Testing security layers + +### Hypothesis, test case, counter example + +The goal of a good tester is to test hypotheses. A hypothesis is just a +scientific way of asking a question. The hypothesis of this assignment is "This +security layer is well designed." The questions you will ask when running your +test cases will always be the same + + * "Is this reference monitor secure?" + + * "Does this reference monitor hamper performance?" + + * "Does this reference monitor prevent actions that should be allowed?" + +Notice that these questions are parallels of the security paradigms: security, +efficiency and accuracy, respectively. + +If we can find a case where the hypothesis is false, then the security layer is +not secure. Such a case is referred to as a counter example. Hence all test +cases should be designed to test for these three types of flaws. + +#### Information on: Try, Except, Else, Finally + +The try, except, else and finally statements are part of **exception handling**. +For more information on exception handling please visit: + + * [http://docs.python.org/tutorial/errors.html] + * [http://wiki.python.org/moin/HandlingExceptions] + * [http://www.tutorialspoint.com/python/python_exceptions.htm] + +### Hints and Ideas for testing + +When writing your own tests it is important to test for a complete set of +possible penetrations. Keep in mind, it only takes one test case to break +through a security layer. Some of the things you may want to test for include: + + * threading + * correct error handling + +And more! Remember a good security layer can't be broken by anyone! Which is +all part of the fun! It's about solving a puzzle. First you make the puzzle - +write the security layer, then you solve the puzzle - try to bypass it. If your +puzzle is "good enough", no one will be able to break it, no matter what. + + +> Note: When creating tests, the best practice is to separate them into +> different files. Each test file should focus on testing one specific scenario +> or functionality. If there is a flaw in any part of a test file, we will +> discard the entire test file. This modular approach helps isolate issues - if +> there is a flaw in one test file, it will not affect or invalidate the other +> test files, since each file is independent and testing distinct functionality. + + +## Notes and Resources + + * The following link is an excellent source for information about security + layers: https://ssl.engineering.nyu.edu/papers/cappos_seattle_ccs_10.pdf + + * In repy `log` replaces `print` from python. Many students find this to be a + stumbling block. + + * Note that you should not assume that any files exist in your directory. You + should create any files (e.g., `testfile.txt`) yourself in your test program. + + * It's important to note that if a test has a flaw in any part of it, the + entire test will be considered invalid. So, it's advisable to break your + tests into different files. + +## How to run your tests on many reference monitors + +Create a directory that the security layers will write their files into. You +need to run repy with only access to this directory. You can write a test +program that does `log(str(listfiles()))` to see if you are in the right place. + +Have all the reference monitors to test and the test cases inside the same +directory where the `repy.py` file exists. + +* In the bash shell on Mac and Linux: +```bash +for referencemonitor in reference_monitor_*; do for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` + +* In the Command Prompt on Windows: +```cmd +FOR %r IN (reference_monitor_*) DO @FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a +``` + +* In PowerShell on Windows: +```powershell +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the output from each program. Make sure that you replace +`` with your NetID. + +If you want to spot the reference monitor that failed during the test run, add +echo the name of each reference monitor before the inner loop, like so: + +* In the bash shell on Mac and Linux: +```bash +for referencemonitor in reference_monitor_*; do echo $referencemonitor under test; for testcase in _*; do python repy.py restrictions.default encasementlib.r2py $referencemonitor $testcase; done; done +``` + +* In the Command Prompt on Windows: +```cmd +FOR %r IN (reference_monitor_*) DO @(ECHO %r under test & FOR %a IN (_*) DO @python repy.py restrictions.default encasementlib.r2py %r %a) +``` + +* In PowerShell on Windows: +```powershell +foreach ($referencemonitor in Get-ChildItem reference_monitor_*) { Write-Host $referencemonitor.Name; foreach ($testcase in Get-ChildItem _*) { python repy.py restrictions.default encasementlib.r2py $referencemonitor.Name $testcase.Name } } +``` + +This will print out the name of each reference monitor before it starts +executing the test cases against it. + + +## What to turn in? + +* Turn in all the test cases in a zip file. The name of each testcase must match the following format: `_attackcase.r2py`. For example, if your netid id `abc123`, then the submitted zip files must contain a set of files named as `abc123_attackcase1.r2py`, `abc123_attackcase2.r2py`. There are no restrictions on the number of attackcases that can be submitted. + +* **Never raise unexpected errors or produce any output.** Your attackcase must not produce any error or output, if the reference monitor is behaving as it should. This applies for error checking as well. If the attackcase tries to trigger an expected error, it should catch and suppress that error if the reference monitor correctly raises that error. + +* While you're allowed to test if the correct errors are being raised, you aren't allowed to check the error message for any errors. That means checking for `FileInUseError` is valid, but it's not valid to check the specific error message. + +* Attackcases should be independent of each other. This means that you should not try to use any file (or it's contents) across different attackcases. \ No newline at end of file From 06c34f68364fc95aaa5fba92ccc9912ad857aed3 Mon Sep 17 00:00:00 2001 From: vgharini Date: Tue, 21 Oct 2025 19:45:09 -0400 Subject: [PATCH 122/122] ImmutableVersionThree --- .../ImmutableVersionThree.md | 97 +++++++++++++++++++ 1 file changed, 97 insertions(+) create mode 100644 EducationalAssignments/ImmutableVersionThree.md diff --git a/EducationalAssignments/ImmutableVersionThree.md b/EducationalAssignments/ImmutableVersionThree.md new file mode 100644 index 00000000..2c586db1 --- /dev/null +++ b/EducationalAssignments/ImmutableVersionThree.md @@ -0,0 +1,97 @@ +# Fixing your security layer + +In this assignment you will analyze the bugs in your security layer from [Part +One](./ImmutableVersionOne.md) +and fix them. You may want to use test cases from [Part +Two](./ImmutableVersionTwo.md) +to help identify these bugs. + + +## Overview + +In this assignment you are fixing your reference monitor. You have been given a +bunch of test programs that try to compromise your reference monitor. Your job +will be to determine where your reference monitor failed, and fix it to ensure +an attacker cannot circumvent the security layer. + +## Code Analysis + +Given below is a section taken from the sample reference monitor given +in [Part One](./ImmutableVersionOne.md#a-basic-and-inadequate-defense). + +```py +class VMFile(): + def __init__(self, filename, create): + # globals + # If a file with the same 'filename' already exists, this creates a new version 'filename.v1'. + # (Incomplete: does not handle further versions like v2, v3, etc.) + if create: + if filename in listfiles(): + # File exists → create version 1 + # bug ? + prev_file = openfile(filename, False) + content = prev_file.readat(None, 0) + + new_name = filename + ".v1" + self.VMfile = openfile(new_name, True) + self.VMfile.writeat(content, 0) + else: + # File doesn't exist → create filename + # bug ? + self.VMfile = openfile(filename, True) + else: + # Open existing file normally + self.VMfile = openfile(filename, False) + + def readat(self, num_bytes, offset): + # bug ? + return self.VMfile.readat(num_bytes, offset) + + def writeat(self, data, offset): + # bug ? + return self.VMfile.writeat(data, offset) + + def close(self): + return self.VMfile.close() + + +def LPopenfile(filename, create): + # bug ? + return VMFile(filename, create) + +def LPremovefile(filename): + removefile(filename) + +def LPlistfiles(): + # bug ? + return listfiles() +``` + +Let's analyze some of the bugs in this implementation (not in any specific order). + +1. As outlined in the specifications in [PartOne](./ImmutableVersionOne.md), if the openfile() is called on an already open file, then the relevant error (`FileInUseError`) must be thrown. This scenario is not correctly handled in the sample implementation. + +2. Similarly, if the create=True is used on a versioned file, it should raise RepyArgumentError("Cannot create explicit version files"), as manual version creation is not allowed. + +3. The reference monitor needs to track the state of the information on disk, but cannot re-read it for every access (due to efficiency concerns). A common mistake is when the attacker can cause the reference monitor’s state to diverge from the underlying system’s state, especially in error conditions. The reference monitor and disk's states should be in sync. + +4. Race conditions and Time-of-check-to-time-of-use (TOCTTOU) issues are frequent oversights. Attackers may exploit timing gaps between checks and operations to corrupts state or bypass restrictions. Such bugs can cause inconsistencies or even infinite loops in the reference monitor. + +5. Some reference monitors inappropriately share state for different files. An attacker may be able to exploit this and cause security and accuracy issues. + +6. Many reference monitors had accuracy or security bugs as a result of not properly following the instructions. + +7. Finally, this example doesn't use locks, which makes it vulnerable to multi-threaded race conditions - effectively the same root cause as TOCTTOU problems but arising from concurrent access. + + +## What to turn in? + +* Submit 2 files: + 1. 1 r2py file: Your reference monitor should be named as `reference_monitor_[netid].r2py` with all +letters in lowercase . For example, if your netid is `abc123`, the file should be named: `reference_monitor_abc123.r2py`. + 2. 1 pdf file: Write up a 1 page report explaining the rationale behind the vulnerabilities and how you fixed them. Name it `[netid].pdf`. + +* **Never raise unexpected errors or produce any output.** Your reference monitor must +produce no output when run normally, with a valid attack case. Make sure that you remove any `log()` statements used for debugging before submission. + +* You are required to modify your reference monitor so that it generates no output (logs or errors) when any valid attack case (from Part 2.2, after invalid attack cases have been identified and excluded) is executed against it.