Securing your Big Data

Securing your data from both internal and external breaches is increasingly important both in terms of the cost of loss of the initial data and subsequent reputational damage.

Securing your data can be grouped in to 2 broad categories:-

Preventative measures

The following areas should be covered to provide a robust security solution:-

User authentication

Ideally, you need a site-wide authentication system where user and group information can be maintained centrally. Each application in your organisation should be able to use a “single sign on” (SSO) so that users don’t have to re-enter their details for each application. For enhanced user authentication security, you can use techniques such as:-

1. Captcha (which prevents external attacks by computers designed to generate random usernames and passwords to try to gain access)

2. 2-stage authentication. This is the type of authentication that you’ll get for your bank account, where you have to not only enter your username & password, but also enter, for example, a generated key code which is updated each minute.

Authorisation (Data & Reports)

Once a user is authenticated, data & reports need to be secured so that only certain users have access to that information. Typically, users are assigned to groups or roles, and the authorisation privileges are then assigned to the group/role. Privileges can allow users to create, read, update and/or delete data, dependent on what they’ve been assigned to.

Dynamic Data Masking

Dynamic data masking refers to the scrambling, masking or removal of data at a row or column level from a query result set.

Row level security

As well as user’s being authorised to see data at a table or report level, privileges can be assigned to certain rows of data. For example, if you have a database that contains customer data from all over the world, but for legal reasons only greek users can see greek customer data, then a row-level filter can be applied to prevent non-greek users seeing any customer data whose country of residence is Greece.

Column level security

In a similar fashion, column level security can be imposed so that sensitive data is not presented to users who are not entitled to see it. For example, your internal investigations team may want to see sensitive customer payment card details, and the sensitive data may be located in columns within a table that also holds non-sensitive data that would be useful to the rest of the organisation. In this situation, you need to be able to redact/scramble sensitive data when non-investigations staff run a query against the customer payment card table.

Approval process

For user authentication & authorisation to work properly, you will need a business process in place that ensures that approval isn’t granted without necessary stakeholder involvement.

Network Encryption

Another situation where data breaches are prevalent are in the transmission of sensitive data including username and password across unencrypted networks. By using secure network protocols which utilise encryption, this can be prevented.

Database, export & backup encryption

To avoid situations where a username or password may have been obtained or data has been exported or backed up to tape, extra security can be imposed by encrypting data at rest within your database, any exports and any backups.

Take care to ensure that your solution can cope with indexing and foreign key constraints and that the decryption process does not significantly affect performance.

Ensure that your master encryption key is stored securely, otherwise can’t gain access to data or backups!

Permanent Data Masking

This will be required when you move production data to a non-production environment. Production environments generally have tight access controls with only a small number of vetted individuals being given access, whereas non-production environments typically have a very open access with 3rd party suppliers coming on and off the project frequently.


User authentication and authorisation to data should be periodically reviewed, so as to prevent users still having access who have left or changed roles within the organisation.


Although the preventative steps should prevent unauthorised access to your data, you should also put in to place processes that will allow you to detect user activity so that you can identify culprits and what they have been able to do with your data.

User authentication logs and reporting

Repeated password failures should be logged and reported to the security team

User activity (audit) logs and reporting

All read and write activity which occurs against the data should be logged.


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s