Previously on injections…
Before we jump into it, let’s start with a quick recap of the previous post. It is well worth reading to get an overview of injections in general.
Here is the gist of it.
Anything that has structure and is manipulated by code that is unaware of that structure is injectable. Injection happens when the code used to manipulate data does not know its structure. See here for examples.
The above statement practically shouts for a simple and elegant solution.
Make our code aware of the structure it manipulates.
And that’s precisely what we’ll talk about in the rest of this post.
The known-knowns
It is essential to scope this “being aware” properly. Failing to do so will lead you to a rigid and insecure design.
Here is the golden rule for scoping.
Deal with data manipulation within a single component that is as small as possible (a class, a library, a module, etc.) and use this component exclusively from the rest of your codebase.
This idea fits very nicely with OO’s encapsulation concept. It suggests bundling data and methods that work on that data into a single class. Doing so also demonstrates the single responsibility principle. The change of the underlying data structure should only be reflected in this class.
Here is the security version of the SRP.
Code manipulating data has complete responsibility for it. It must not trust other components when it comes to the processing of that data.
Our code must know and be fully in control of the syntax of the data. It must not allow external components to directly modify this syntax.
Unless, of course, this is a requirement. In which case, it must enforce controls to avoid giving too much capability to untrusted components.
Enough of the theory; let’s see this in action!
Practice
Let’s get back to our logging example. Here is the code.
log = 'X event: ' + userInput
logFileHandle.writeLine(log)
...
logs = logFileHandle.readAll()
uiComponent.addItems(logs.splitBy('\n')
The structure of our data is simple lines separated by \n. This is where we need to take special care as a line-feed character breaks the syntax of our logline. These special characters are often referred to as control characters.
When it comes to dealing with syntax violations, there are two options. You can either skip the characters causing the breach and ignore the information they provide. Or escape them to make it compatible.
Option 1, filter
This is as simple as ignoring the given character. Here’s an example in our logging “framework”.
log = 'X event: ' + userInput.replace('\n', '')
// for "malicious\ninput" yields "maliciousinput"
It doesn’t get much simpler than this.
Option 2, encode
Encoding is changing the representation of the data. Variations range from using a simple substitution table to encoding everything as hex-bytes. Let’s see a couple of examples.
Static replacing of control characters
malicious\ninput --> malicious_input
This works well, but we can never be sure if a _ was originally a \n, as these are indistinguishable in the log files. That’s a small price now, but losing data might become a problem if you have multiple control characters.
Hex encoding control characters
malicious\ninput --> malicious\x0Ainput
This is somewhat better as different characters are encoded differently. The previously described problem still exists but in a smaller scope. Hint: how would you differentiate between \x0A and \n in the log files?
So what to do to keep all the information but avoid injection?
Base64 encoding the complete payload
malicious\ninput --> bWFsaWNpb3VzCmlucHV0
This works well, as base64 has no particular control characters. It is essentially a raw bitstream converted to printable characters. Since everything is encoded equally, doing a base64 decode yields the original value.
A more complex approach
To solve the indistinguishability problem. We could introduce a “special” character that precedes and closes every encoded data. Let’s take HTML’s example. It uses the ampersand ("&") for the beginning and the “;” character for marking the end.
< => <
> => >
& => &
> => &gt;
Note that the “&” character is also encoded. This makes it possible to differentiate between original > data and the encoded &gt;.
These are usually referred to as “entities”.
Get more insight with SCADEMY courses
In our courses we start from theory and dive into practice. You have a chance to exploit, analyse and deconstruct applications.
Take a look at our open sessions
The price of encoding
If you encode, then you might also need to decode before viewing or processing the data. Depending on how many encodings you did, this may be skipped altogether.
Encoding works pretty well. Now let’s take a look at something that has more syntax than a single character.
A complex structure
In a more complex structure, like HTML, filtering is hell of a lot harder. Think about dropping the tags from this input:
malicious <b>BOLD</b> inputSimple, replace to the rescue!
input.replace('<b>', '').replace('</b>', '') // yields "malicious BOLD input"All good, right? Now check this!
malicious <b<b>>BOLD</b</b>> input
And after our secure filtering:
malicious <b>BOLD</b> inputSo what, I just do the regex twice, or three times or better yet, n times! How about that?!
You and I both know n needs to be limited to avoid denial of service attacks. So this doesn’t work perfectly. Libraries taking this approach usually implement some kind of depth limit on escaping. If this limit is reached and data is still not clean, they throw an error.
Another option is to implement a state machine and parse the input that way. That works very well, and you will quickly reach a point where you are essentially rewriting parts of the original parser that interprets the data.
And guess what? You have to treat every edge case like the original parser does! Otherwise, a malicious payload might slip through.
And, if it so happens that you need to keep some parts of the data while encoding others, you are in for a treat. That is a more complex undertaking. We’ll talk about that in another post.
As a golden rule for now:
Stick with encoding!
Okay, so where do I encode? Remember this from the beginning?
Deal with data manipulation within a single component that is as small as possible…
The encoding logic must be encapsulated within the object that deals with the data. Ideally, it provides an interface others can rely on without knowing the structure of the underlying data.
Let’s look at an example.
The SQL driver
Here is a question:
Who is the ultimate owner of a SQL query? In other words, who should do the escaping?
Is it the application? The driver? The database?
Any of these might be a valid answer. I argue that the ultimate owner is the database server. The DB, by default, gives you access to the internal representation of queries. This is the original interface.
Injection happens when you get smart and construct these queries without having tools that fully understand the SQL syntax. Okay, how are you supposed to query data then?
Use prepared statements and let the SQL engine deal with data binding.
This brings us to the silver bullet against injections.
Data binding is king
When you bind data, you let the owner of the data structure deal with all the escaping and construction. Let’s take a peek at how prepared statements work on a high level. Take the following call to the SQL driver.
client.query('SELECT * FROM table WHERE c1 = ? AND c2 = ?', input1, input2)
It will essentially translate into multiple instructions on the SQL server.
Driver => DB: compile this statement 'SELECT * FROM table WHERE c1 = ? AND c2 = ?'
Driver => DB: execute statement with input1, input2
Now compare this with trying to manually escape things.
client.query('SELECT * FROM table ' +
'WHERE c1 = \'{escape_string(input1)}\' AND c2 = {escape_int(input2)}')
Notice how much extra knowledge your code needs to have about the underlying structure. input1 is a string, so I need some code that makes sure it is safely handled as a string in SQL with escape_string(). input2 is an integer, so I need to make sure it is dealt with as an integer using escape_int().
The implementation details of these methods are not essential. What’s important is that the detailed structure of the underlying data is exposed to your code.
At this point, you might argue that there is no extra exposure as you already need to know that in table, c1 is a string and c2 is an integer. True, but that isn’t the problem.
The problem is that you need to deal with the peculiarities of how SQL treats strings and integers and syntax violating characters in them.
With a well-designed interface, you can completely hide this knowledge from your consumers and make it impossible or very hard for them to make mistakes.
Interfaces are security contracts!
I’ll expand on this idea in another blog post.
Out of ammo, can’t bind?
There are some cases where you need dynamic data at places where you can’t bind. For instance, in SQL, these could be table names.
SELECT column1,column2 FROM <table_name> WHERE id = 8;
Databases and drivers don’t provide an interface to bind the table name. Some libraries act like they do, but that’s an illusion. It is, however, worth looking at what they do under the hood.
Well-written libraries do rigorous checking of such dynamic data. For table names we are lucky as they are easily checked with regex, for instance: ^[a-zA-Z][a-zA-Z0-9_]{0,30}$. The actual regex may vary across engines. What’s important is the simplicity and the allow-listing approach. It defines what’s normal and doesn’t try to match violations.
If you can, use very strict allow-listing. If there are only a couple of possible values, list them out one by one. If the pattern is only part dynamic, use regexes with beginning and end matching.
Why allow-list instead of block-listing?
There are a couple of reasons. First, when you try to block things, you have to adjust your defenses every time a new attack comes out. It is more expensive to maintain.
Second, make a mistake in allow-listing, and you block something that should be allowed. It’s an inconvenience, but no data was lost, and the legit user will likely report the error. Have it the other way around, and you are in for a treat.
Summing it all up
Injections can occur when the peculiarities of data structures are exposed to a code that is unaware of them. As a golden rule:
Deal with data manipulation within a single component that is as small as possible and use this component exclusively from the rest of your codebase.
Always prefer using interfaces that support data binding. Push the responsibility of manipulating data as close to the owner of the data as possible. A great example is the SQL engine and its prepared statements interface.
Another strong example is HTML’s .innerText. It leaves no room for error. Anything assigned to innerText will be insterted as text into the DOM. These interfaces combined with Trusted Types may finally maybe put an end to our
XSS troubles.
If you can’t bind, use strict allow-listing. Specific values work best; if that’s not an option, use regex with ^ and $ matching.
Finally, always think of interfaces as security contracts.