Storing Passwords in a Database

In this post, I'll go through three common ways to store and retrieve passwords in a database. I'll assume PHP and MySQL, but the techniques should be very similar for other setups.

Only the last method should ever be used for security reasons, but unfortunately, a large number of sites use one of the less secure methods and put their users in danger.

Storing a Password as Plain Text

This is the most basic and definitely least secure method for handling passwords. Never Use This Method!

The basic strategy is to store the password directly in the database. You would then authenticate a user by running a query like this:

SELECT id FROM users WHERE username='johnsmith' AND password='123456'

If the query returns a row, the username and password are correct. As you can see, if anyone intercepts this query along the way, they automatically have the user's username and password. Also if your database gets stolen, the thief has all of your users' usernames and passwords. This is an even bigger problem because most people use the same username and password for everything.

If you ever click a Forgot Your Password link and the site gives you your current password, they are using this method and I would highly suggest not using the site or at least using a unique password just for the site.

Using a Password Hash

This method is better than plain text, but still has some major, relatively little known, security holes in it. A lot of sites use this method thinking it is secure. Again, do not use this method.

This method takes a user's password and converts it to an md5 or similar hash before storing in the database. For example, "123456" becomes "e10adc3949ba59abbe56e057f20f883e". This seems like it solves the problem with plain text since a person cannot look at the hashed string and know the user's password. But, if you type this hash into Google, the second result is titled "Google Hash: md5(123456) = e10adc3949ba59abbe56e057f20f883e". Not as secure as it first looked, is it?

There are md5 hash tables you can download that contain every word in the dictionary and every common password that make this method very susceptible to attacks. Many sites require passwords with numbers, symbols, capital letters, etc., which helps fix the security hole, but why make things harder on your users when you can just use a password salt?

Using a Password Salt

There is no reason not to use this method. It provides an extra layer of security on top of a password hash with very little extra work.

This method generates a random string (salt) and appends it to the user's password before generating an md5 or similar hash. Then, both the password hash and the password salt are stored in the database and used to authenticate the user. For example, "123456" becomes "123456ghjfdweurt" becomes "8e1a92e8f87a5bbf36f26e330cf7f0b5". Try typing that hash into Google and the most you may find is this article.

Here's the PHP code for initially inserting a user into a database. The getRandomString() function is from http://www.lost-in-code.com/programming/php-code/php-random-string-with-numbers-and-letters/.

//get username and password
$username = $_REQUEST['username'];
$password = $_REQUEST['password'];

//generate password salt
$password_salt = genRandomString();

//generate password hash
$password_hash = $password . $password_salt;

//insert into database
$query = "INSERT INTO users (`username`,`password_hash`,`password_salt`) VALUES ('$username', '$password_hash', '$password_salt')";
mysql_query($query);



function  genRandomString() {
    $length = 10;
    $characters = ’0123456789abcdefghijklmnopqrstuvwxyz’;
    $string = ”;    

    for ($p = 0; $p < $length; $p++) {
        $string .= $characters[mt_rand(0, strlen($characters))];
    }

    return $string;
}

Here's the code for authenticating a user once they are already in the database:

//get username and password
$username = $_REQUEST['username'];
$password = $_REQUEST['password'];

//query database
$query = "SELECT * FROM users WHERE username='$username'";
$result = mysql_query($query);

//if no result, username is incorrect
if(!$result) {
    //authentication failed
}

//get database row
$row = mysql_fetch_assoc($result);

//generate password hash from entered password
$password_hash = md5($password . $row['password_salt']);

//check if the generated hash is equal to the hash in the database
if($password_hash === $row['password_hash']) {
     //authentication passed
}
else {
     //authentication failed
}

Important Safety Tip

No matter what method you use, SSL encryption is essential during authentication to protect against man-in-the-middle attacks. This is where an attacker intercepts data between the user and the server. If the user submits a login form and an attacker intercepts it, the password will be compromised no matter which method you use.

Installing and Configuring Dornbase

Dornbase is a PHP contact management system I started a while ago and recently released via Sourceforge (http://sourceforge.net/projects/dornbase/). You can see a demo of it running at http://jeremydorn.com/dornbase.php.

The project is still in its early stages, so there is not much documentation available. I plan to eventually implement an automated installer, like Joomla and most other major PHP applications have. If you have any suggestions or want to help, email me at jeremy@jeremydorn.com.

This post is meant to serve as a guide for installing and configuring Dornbase on either your local machine or a remote host.

Minimum Requirements

Most paid web hosts meet these requirements. All of the required software is included in xampp (http://sourceforge.net/projects/xampp/).

  • Apache 2.2 with mod_rewrite enabled
  • PHP 5+ (make sure ERROR_REPORTING is set to not show E_NOTICE)
  • MySQL 5+ (I haven't tested older versions, but they probably work too)

Download Files

Download Dornbase 1.7 from http://sourceforge.net/projects/dornbase/ and extract the files to your document root (C:/xampp/htdocs/ if using xampp).

Set up Database

Create a new database and run all of the queries in the db_setup.sql file included in the download to set up the database tables. If you are using xampp, follow the phpmyadmin instructions below to do this.

PhpMyAdmin Instructions
  1. Go to http://localhost/phpmyadmin.
  2. In the "Create new database" form, enter a name for the database (e.g. "dornbase") and click the "Create" button.
  3. Click the "import" tab.
  4. Select the db_setup.sql file and click "Go".

The only table you may want to change is the "fields" table, which contains all the fields a record will have. By default, it is set up with all of the fields in the demo, but this can be easily changed. Below is a brief description of what each column in the fields table means.

name- The id of the field (no spaces, all lowercase) (e.g. fname, address_line_1) display- The display name of the field (all lowercase) (e.g. "first name", "address line 1")
datatype- The datatype of the field. Either 'text', 'number', 'date', or 'boolean'.
type- A more specific type differentiator (e.g. 'phone', 'website', 'money', 'ssn')
multiple- Can the field have multiple values? '1' for yes, '0' for no
discrete- Are the field values limited and discrete (i.e. will it use a drop down menu)? '1' for yes, '0' for no
search- Should this field be searchable? '1' for yes, '0' for no
default- The default value this field will have for a new record
summary- Is this field vital for uniquely identifying a record?
padto- For numeric fields, the max number of digits (e.g. if '12', all values are left padded with 0s until there are 12 characters left of the decimal point). This enables storing numbers as text and still being able to properly sort them.

Configure Server

Rename the "server_default" folder to "server". This folder contains 2 files which you need to edit. There are comments and instructions in each of the files.

Rename "htaccess-default" to ".htaccess" (You may need to open it in notepad and use Save As on some Windows machines). Edit the line "RewriteBase /dornbase" and change "/dornbase" to whatever directory you installed the application in.

Rename the "config_default" folder to "config". This folder contains several files that help you customize Dornbase to fit your needs. I'll explain each of these files later.

Test it out

Open up Dornbase in a browser (if you installed it in /htdocs/dornbase/, go to http://localhost/dornbase/). You should see a sign in page. Sign in with the default admin account (username: "admin", password: "password"). If you see an empty record page, everything is installed correctly.

You may see errors if you modified the record fields in the database. After the next step, this should be fixed.

Customize

The important files to edit in the config folder are template.php, record.php, and functions.php.

template.php is really short and defines a few variables such as business name and return address (for printing envelopes).

record.php only needs to be changed if you modified the record fields in the database at all.

functions.php is a large file and contains a bunch of callback functions that provide hooks into the application. I'll go through a few of the important functions.

  • get_record_header(&$data) - Should return the header for a record. By default, it displays the full name (or company if name is empty) followed by a description and the policy type.
  • get_field_order(&$record, $mobile=false) - This determines the layout of the record. It allows you to layout the record differently depending on the record data. For example, you can have a "spouse name" field only show up if their status is set to "married". You can also display different fields on a mobile device.

You can play around with the other functions and the other files in the config folder if you want.

After you make a change to a file in the config folder, most of the time refreshing the page will work. In a few cases, you may have to sign out and sign back in for the changes to take effect.

Reducing Javascript and CSS Load Times

I was making a complex ajax web application and ran into problems with load times. In some cases, it was taking up to 7 or 8 seconds to load a page. I found out the culprit was multiple large javascript and css files. Here's how I was able to get my load times under a second with a few changes. All of the examples below are for javascript files, but they apply equally to css files.

Combine javascript and CSS files

There is overhead each time the browser has to load an external file, so reducing the number of server calls can speed up page loads. Originally, I put all of my javascript scripts in one large file. This helped page load times, but made it difficult to edit individual scripts. I then switched to a PHP solution.

I created the following PHP page, which combines multiple javascript files on the fly.

<?php
//set content type
header('Content-type: text/javascript');

//pull list of comma-separated files from $_GET
$js_files = explode(',',urldecode($_REQUEST['files']));

//read each file
foreach($js_files as $js_file) {
 echo "\n\n\n/*******************$js_file********************/\n";
 $file = '/path/to/js/directory/'.$js_file.".js";
 if(file_exists($file)) readfile($file);
}
?>

Here is how you would load javascript files in the head section of page.

<!-- Using PHP's urlencode function -->
<?php
$files = array("jsfile1","jsfile2","jsfile3");
?>
<script type='text/javascript' src='/path/to/js/directory/javascript.php?files=<?php
 echo urlencode(implode(',',$files));
?>'></script>

<!-- No PHP, hard code commas as "%2C" -->
<script type='text/javascript' src='/path/to/js/directory/javascript.php?files=jsfile1%2Cjsfile2%2Cjsfile3'></script>

This reduced my load times to about 5 or 6 seconds.

Take Advantage of Parallel Downloading

This partially contradicts the previous tip when I said less files were always better. Most browsers are capable of downloading a limited number of javascript and css files at the same time. For example, Firefox will usually download up to 3 files simultaneously from a single host.

If you have 1 or 2 large javascript files, you should put them in their own script tags and combine the rest of the scripts as before. This way, the browser doesn't have to wait for the large script to download before fetching the other ones. Here's an example.

<!-- Large js file -->
<script type='text/javascript' src='/path/to/js/directory/javascript.php?files=largefile'></script>

<!-- Multiple smaller files -->
<script type='text/javascript' src='/path/to/js/directory/javascript.php?files=jsfile1%2Cjsfile2%2Cjsfile3'></script>

Even though you are only loading one file in the first case, you should still use the php script to take advantage of the later tips.

This reduced my load times further to about 2 or 3 seconds.

Compress Javascript and CSS files

The basic idea behind this is to reduce the size of javascript and css files by removing comments and extra white spaces among other things. There are two ways to do this, each with their advantages and disadvantages.

The first is to compress the files directly. The advantages here are speed and reduced server load. The server doesn't have to waste time compressing each file at runtime. The big disadvantage is that compressed files are almost impossible to edit. This method works best for 3rd party scripts, such as javascript frameworks, that you aren't going to be editing yourself. Most frameworks offer compressed versions directly, but you can also compress any file yourself with http://jscompress.com/ for javascript files and http://www.cssdrive.com/index.php/main/csscompressor/ for css files.

The other method is realtime compression. The advantage is files remain editable and the development process is much easier. The disadvantage is that the server must re-compress each script at runtime, which takes time and processing power. This method is best for scripts which you edit often. There is a great php script for doing this called minify that works for js and css files. They have plenty of documentation on the site and it's pretty easy to implement into the php script we created above.

Compressing files can reduce the size by anywhere from 10% to 50% or more. As an example, the uncompressed jquery framework is 160kb and the compressed version is 70kb. This can potentially cut loading times in half, but the actual gain is probably less than that.

Take Advantage of Caching

A lot of times, javascript and css files don't change very often and it doesn't make sense to load them on each page view. With a few simple changes to our php script, we can make the browser cache these pages. This provides the biggest improvement in load times.

<?php
//set content type
header('Content-type: text/javascript');

//if the nocache parameter is not explicitly set
if(!$_REQUEST['nocache']) {
 //store in cache for 1 day (86400 seconds)
 header('Expires: '.gmdate('D, d M Y H:i:s', time()+86400).'GMT');  
 header('Cache-Control: max-age=86400'); 
 header('Pragma: max-age=86400'); 
}

...

If you have some file which changes frequently or is generated at the server side, you can pass "nocache=true" in the script tag source.

One problem you may notice is that if you make a change to a javascript file, a user may not get the latest version for a full day, which could break the site in the meantime. Luckily, there is an easy solution.

If the script src changes, a browser will always reload the script, even if it was cached. Since we're already passing parameters in the script url, we can add "version=#" and increment this number whenever we make a change. That way, the script will be reloaded the next time the page is.

The first time the webpage is loaded, there is no added benefit, but the load times of subsequent pages was reduced to less than a second.

If you can think of any other tips to reduce page load times, let me know.

Fixing jQuery UI Autocomplete

jQuery UI 1.8 was recently released and it included a much needed autocomplete widget. For the most part, this widget works fine, but there is an annoying bug that makes it not quite ready for a production environment. Basically, if a user clicks on an option and doesn't release the mouse button fast enough, the option is not selected. You can see the bug in action on the jQuery UI autocomplete demo.

Thankfully, there is a workaround.

Here is what we would like to be able to do:

$("#autocomplete").autocomplete({
   source: 'search.php',
   select: function(event, ui) {
      alert(ui.item.id);
   }
});

Here is the bug-fixed version I came up with:

//global variable that stores the last focused option
var search_option = false;

$("#autocomplete").autocomplete({
   source: 'search.php',
 
   //when an item is focused, store item in the global variable
   focus: function(event, ui) {
      search_option = ui.item;
   },
 
   //when an item is selected with the keyboard, trigger
   //the mouse down event for consistency
   select: function(event, ui) {
      search_option = ui.item;
      $("#autocomplete").autocomplete('widget')
      .trigger('mousedown.choose_option');
   }
})

//bind the select event to mousedown
.autocomplete('widget').bind('mousedown.choose_option',function() {
   //immediately closes autocomplete when option is selected
   $("#autocomplete").autocomplete('close');
 
   //perform desired action
   alert(search_option.id);
});

Obviously, this isn't a very elegant solution, but I didn't want to mess around with the actual autocomplete code. Hopefully, this is fixed in the next release. Until then, this will have to do.