SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512

NEW 07 JANUARY 2004 Read on below for information about the 1.1 version that includes SHA-1, SHA-224 functionality.

Here's my open source implementation of the SHA-1, SHA-224, SHA-256, SHA-384, and SHA-512 secure has algorithms. It's released under an open source BSD license. The README contained in the tarball below is included later on this page. The tarball contains seven files, the README file, the SHA-1/224/256/384/512 implementation in two files, sha2.c and sha2.h, and several utility programs, sha2test.pl and mktestdata.pl (two perl scripts for testing to make sure the library is working correctly and comparing with known test vectors), sha2speed.c (to test how fast the implementation is working), and sha2prog.c, a utility that uses the library to generate SHA-224, SHA-256, SHA-384, and SHA-512 hashes from either standard input, or from a file.

HMAC-SHA1: Before writing my SHA implementations, I wrote an implementation of the HMAC-SHA1 algorithm (it includes Steve Reid's public domain SHA1 implementation since I wrote it before adding SHA-1 to my own library). I wrote it initially as part of a demonstration of Prof. Ron Rivest's Chaffing and Winnowing technique. One evening I dredged up the old code and cleaned a few things up and tarred and gzipped it. My code is released under a BSD-style open source license, while my changes to Steve Reid's SHA1 code are in the public domain (as his code is). If you have an SHA-1 implementation in a library, you can use that instead (or have it use my SHA-1implementation). There's no documentation except in the source files themselves. It was compiled and tested quite some time ago under FreeBSD 4.0 and on Linux (at roughly the same time FreeBSD 4.0 was relatively new). Thanks to Compaq (now HP), and their multi-platform test-drive site, it's even been built and minimally tested on Compaq's Tru64 Unix running on Intel platforms and Compaq Alphas. I make no guarantees about the accuracy, usefulness, or anything else of the code. Use at your own risk.

hmac_sha1.tar.gz

SHA-256/384/512

The current stable (and fairly well tested) version of SHA-256/384/512 is version 1.0.1, but it does not include SHA-1 or SHA-224 functionality.



The latest version is 1.1 adds SHA-1 and SHA-224 functionality. However, the new version does consolidate the SHAxxx_CTX data structure into a single SHA_CTX structure (union), which means a minor, quick edit to anything interfacing with the implementation using the 1.0 style interface. Be aware that SHA-1 is considered broken and may not be appropriate in some applications.

A subdirectory with 20 test vectors is included. The perl scripts can be used (at least on FreeBSD systems they work) to make sure that the program is working correctly. Included are the NIST test vectors, as well as several that other implementors have shared with me, or I've found on the Internet. I have also included some of my own. So far, this implementation appears to agree with output from other versions I've encountered.

Thank you, David Ireland for your additional vectors. Thanks also to Stefek Zaba, whose SHA256 implementation in POP11 appears to agree with my C implementation. Thanks to Tom St Dennis for some additional test vectors, including some SHA-224 vectors.

Thank you, Rogier van de Pol, for exposing the off-by-one flaw in my beta implementation in versions earlier than 1.0 RELEASE.

Thanks to Stephane Leon for noting that "MEMSET_BZERO(context, sizeof(context))" emits warnings in some Clang versions. I've updated the code to do "MEMSET_BZERO(context, sizeof(SHA_CTX))" instead.

Thanks to rvkisley for pointing out a problem in the perl test script (and suggesting a fix).

Download the latest stable (well tested) version: 1.0.1 (RELEASE)
sha2-1.0.tgz (Unix tarball, gzip compressed)

sha2-1.0.zip (ZIP archive)


Download the latest untested ALPHA version that includes SHA-224: 1.1.1 ALPHA
sha2-1.1.1-ALPHA.tgz (Unix gzipped tarball)


If you have access to a machine with big endian architecture and you have some spare time to blow, I would appreciate hearing from you about how or if this implemention works, particularly the new 1.1.1 ALPHA version.. If you do email me, please let me know what version you tested.

Thanks to Jun-ichiro itojun Hagino for several suggestions, including the suggested change to use u_intXX_t types since most BSD-based systems support them (and Linux x86 apparently does too, at least on the Debian box I tested the implementation on). Thanks also to those who compiled and tested the version and emailed me.

See NIST's Secure Hash Standard web site. That page links to the PDF document describing the algorithms. Several of the test vectors used by the sha2test program were taken from this document.

See NIST's FIPS 180-2 SECURE HASH STANDARD CHANGE NOTICE for preliminary information about SHA-224.

As of winter 2005, SHA-1 is considered broken. Read Bruce Schneier's blog entry about it for additional information. My library will still include SHA-1, but it is recommended that you not use it.

The below text is from the version 1.1 README file:
VERSION:

This is version 1.1.1 ALPHA

Ignore the next RCS version line, please.  Thanks.
$Id: README,v 1.2 2004/01/05 09:51:12 adg Exp $

The 1.0 RELEASE (the previous version) appears to be quite stable
and has been used in quite a few places.  This new ALPHA version
adds SHA-224 functionality, updates the perl scripts for doing
test vectors, consolidates the various SHAxxx_CTX structures into
a single union SHA2_CTX, and consolidates a tiny bit of code.
The bulk of the hashing program is unchanged, however.  And other
than the consolidation of the data structure, the interface is
identical.

This release is ALPHA because I have not yet done testing on any
other platforms.  I will be testing it on MAX OS X (big endian)
soon.  I welcome other platform tests to work out portability bugs
that may have been introduced.  I also welcome reports of issues
with the new test vector perl script changes.

The SHA-224 implementation appears to agree with those of other
folks who have posted test data on the web (as of 05 JAN 2004), the
very few test vectors I was able to find.


LICENSE:

This implementation is released under an OPEN SOURCE BSD license.


WHAT IT IS:

The files sha2.h and sha2.c implement the SHA-224, SHA-256, SHA-384, and
SHA-512 hash algorithms as described in the PDF documents found at the

  http://csrc.nist.gov/cryptval/shs/sha256-384-512.pdf
  http://csrc.nist.gov/publications/fips/fips180-2/FIPS180-2_changenotice.pdf

The interface is similar to the interface to SHA-1 found in the OpenSSL
library.

The file sha2prog.c is a simple program that accepts input from either
STDIN or reads one or more files specified on the command line, and then
generates the specified hash (either SHA-224, SHA-256, SHA-384, SHA-512, any
combination thereof, or all four hashes at once).


LIMITATIONS:

This implementation has several limitations:

 * Input data is only accepted in octet-length increments.  No sub-byte
   data is handled.  The NIST document describes how to handle sub-byte
   input data, but for ease of implementation this version will only
   accept message data in multiples of bytes.
 * This implementation utilizes 64-bit integer data types.  If your
   system and compiler does not have a 64-bit integer data type, this
   implementation will not work.
 * Because of the use of 64-bit operations, many 32-bit architectures
   that do have 64-bit data types but do operations most efficiently
   on 32-bit words, this implementation may be slower than an
   implementation designed to use only 32-bit words (emulating the
   64-bit operations).
 * On platforms with 128-bit integer data types, the SHA-384 and SHA-512
   bit counters used by this implementation might be better off using
   the 128-bit type instead of simulating it with two 64-bit integers.
 * This implementation was written in C in hopes of portability and for
   the fun of it during my spare time.  It is probably not the most
   efficient or fastest C implementation.  I welcome suggestions,
   how to speed things up without breaking portability, as well as
   suggestions to improve portability.


BEFORE YOU COMPILE (OPTIONS):

Each of the options described below may either be defined in the sha2.h
header file (or in the sha2.c file in some cases), or on the command
line at compile time if your compiler supports such things.  For
example:

  #define SHA2_USE_INTTYPES_H
  #define SHA2_UNROLL_TRANSFORM

Or:

  cc -c -DSHA2_UNROLL_TRANSFORM sha2.c
  cc -c -DBYTE_ORDER=4321 -DBIG_ENDIAN=4321 sha2.c

Here are the available options.  Read on below for a description of
each one:

  SHA2_USE_INTTYPES_H
  SHA2_USE_MEMSET_MEMCPY/SHA2_USE_BZERO_BCOPY
  SHA2_UNROLL_TRANSFORM
  BYTE_ORDER (LITTLE_ENDIAN/BIG_ENDIAN)

* SHA2_USE_INTTYPES_H option:
By default, this code uses u_intXX_t data types for 8 bit, 32 bit, and
64 bit unsigned integer type definitions.  Most BSD systems define these,
as does Linux.  However, some (like Compaq's Tru64 Unix) may instead
use uintXX_t data types as defined by recent ANSI C standards and as
included in the inttypes.h header file.  Those wanting to use inttypes.h
need to define this either in sha.h or at compile time.

On those systems where NEITHER definitions are available, you will need
to edit both sha2.h and sha2.c and define things by hand in the appropriate
sections.

* BYTE_ORDER definitions:
This code assumes that BYTE_ORDER will be defined by the system during
compile to either equal LITTLE_ENDIAN or BIG_ENDIAN.  If your system
does not define these, you may need to define them by hand in the sha.c
file according to the byte ordering conventions of your system.

* SHA2_USE_MEMSET_MEMCPY or SHA2_USE_BZERO_BCOPY
The code in sha2.c can use either memset()/memcpy() for memory block
operations, or bzero()/mcopy().  If you define neither of these, the
code will default to memset()/memcpy().  You can define either at the
command line or in sha2.h or in sha2.c.

* SHA2_UNROLL_TRANSFORM
By defining this either on the command line or in sha2.h or sha2.c,
the code will use macros to partially "unroll" the SHA transform
function.  This usually generates bigger executables.  It CAN (but
not necessarily WILL) generate faster code when you tell your compiler
to optimize things.  For example, on the FreeBSD and Linux x86 systems
I tested things on (using gcc), when I optimized with just -O2 and
unrolled the transform, the hash transform was faster by 15-30%.  On
these same systems, if I did NO optimization, the unrolled transform
was SLOWER, much slower (I'm guessing because the code was breaking
the cache, but I'm not sure).  Your mileage may vary.


PORTABILITY:

The code in sha2.c and sha2.h is intended to be portable.  It may
require that you do a few #definitions in the .h file.  I've successfully
compiled and tested the sha2.c and sha2.h code on Apple's OS X (on
a PPC), FreeBSD 4.1.1 on Intel, Linux on Intel, FreeBSD on the Alpha,
and even under Windows98SE using Metrowerks C.  The utility/example
programs (sha2prog.c, sha2test.c, and sha2speed.c) will very likely
have more trouble in portability since they do I/O.

To get sha2.c/sha2.h working under Windows, I had to define
SHA2_USE_INTTYPES_H, BYTE_ORDER, LITTLE_ENDIAN, and had to comment
out the include of  in sha2.h.  With a bit more work
I got the test program to run and verified that all the test
cases passed.


SUGGESTIONS/BUG FIXES:

If you make changes to get it working on other architectures, if you fix
any bugs, or if you make changes that improve this implementation's
efficiency that would be relatively portable and you're willing to release
your changes under the same license, please send them to me for possible
inclusion in future versions.

If you know where I can find some additional test vectors, please let me
know.


CHANGE HISTORY:

0.8 to 0.9 	- Fixed spelling errors, changed to u_intXX_t type usage,
		  removed names from prototypes, added prototypes to sha2.c,
		  and a few things I can't recall.

0.9 to 0.9.5	- Add a new define in sha2.c that permits one to compile
		  it to either use memcpy()/memset() or bcopy()/bzero()
		  for memory block copying and zeroing.  Added support
		  for unrolled SHA-256/384/512 transform loops.  Just
		  compile with SHA2_UNROLL_TRANSFORM to enable.  It takes
		  longer to compile, but I hope it is a bit faster.  I
		  need to do some test to see whether or not it is. Oh,
		  in sha2.c, you either need to define SHA2_USE_BZERO_BCOPY
		  or SHA2_USE_MEMSET_MEMCPY to choose which way you want
		  to compile.  *Whew*  It's amazing how quickly something
		  simple starts to grow more complex even in the span of
		  just a few hours.  I didn't really intend to do this much.
0.9.5 to 0.9.6  - Added a test program (sha2test) which tests against several
                  known test vectors.  WARNING: Some of the test output
                  hashes are NOT from NIST's documentation and are the
                  output of this implementation and so may be incorrect.
0.9.6 to 0.9.7  - Fixed a bug that could cause invalid output in certain
		  cases and added an assumed scenario where zero-length
		  data is hashed.  Also changed the rotation macros to use
		  a temporary variable as this reduces the number of operations.
		  When data is fed in blocks of the right length, copying of
		  data is reduced in this version.  Added SHAYXZ_Data()
		  functions for ease of hashing a set of data.  Added another
		  file sha2speed.c for doing speed testing.  Added another test
		  vector with a larger data size (16KB).  Fixed u_intXX_t and
		  uintXX_t handling by adding a define for SHA2_USE_INTTYPES_H
		  as well as made a few other minor changes to get rid of
		  warnings when compiling on Compaq's Tru64 Unix.
0.9.7 to 0.9.8  - The bug fix in 0.9.7 was incomplete and in some cases made
                  things worse.  I believe that 0.9.8 fixes the bug completely
                  so that output is correct.  I cannot verify this, however,
                  because of the lack of test vectors against which to do such
                  verification.  All versions correctly matched the very few
                  NIST-provided vectors, but unfortunately the bug only
                  appeared in longer message data sets.
0.9.8 to 0.9.9  - Fixed some really bad typos and mistakes on my part that
                  only affected big-endian systems.  I didn't have direct
                  access for testing before this version.  Thanks to
                  Lucas Marshall for giving me access to his OS X system.
0.9.9 to 1.0.0b1  Added a few more test samples and made a few changes to
                  make things easier compiling on several other platforms.
                  Also I experimented with alternate macro definitions
                  in the SHA2_UNROLL_TRANSFORM version (see sha2.slower.c)
                  and eliminated the T1 temporary variable (the compiler
                  would of course still use internal temporary storage
                  during expression evaluation, but I'd hoped the compiler
                  would be more efficient), but unfortunately under FreeBSD
                  4.1.1-STABLE on an x86 platform, the change slowed things
                  down.
1.0.0b1 to 1.0 RELEASE  Fixed an off-by-one implementation bug that affected
                  SHA-256 when hashed data length L = 55 + 64 * X where X is
                  either zero or a positive integer, and another (basically
                  the same bug) bug in SHA-384 and SHA-512 that showed up when
                  hashed data lengths L = 111 + 128 * X.  Thanks to Rogier
		  van de Pol for sending me test data that revealed the bug.
                  The fix was very simple (just two tiny changes).  Also,
                  I finally put the files into RCS so future changes will be
                  easier to manage.  The sha2prog.c file was rewritten to
                  be more useful to me, and I got rid of the old C testing
                  program and now use a perl script with a subdirectory full
                  of test data.  It's a more flexible test system.

1.0 RELEASE to 1.1 ALPHA - Added SHA-224 functionality, consolidated a bit of
                  redundant code, consolidated the SHAxxx_CTX data structure
                  into a single SHA2_CTX union, made the sha2test.pl perl
                  script much more flexible (to allow easy creation of new
                  and interesting test vectors), though it does now require
                  the mktestdata.pl script be in the same directory.


LATEST VERSION:

The latest version and documentation (if any ;) should always be available
on the web at:

  http://www.aarongifford.com/computers/sha.html


CONTACT ME:

I can be reached via email at:

  Aaron Gifford   

Please don't send support questions.  I don't have the time to answer and
they'll probably be ignored.  Bug fixes, or patches that add something useful
will be gratefully accepted, however.

If you use this implementation, I would enjoy getting a brief email message
letting me know who you are and what use to which it is being put.  There
is no requirement to do so.  I just think it would be fun.


EXAMPLES:

Here's an example of compiling and using the sha2 program (in this example
I build it using the unrolled transform version with -O2 optimizations),
and then running the perl testing script:

  cc -O2 -DSHA2_UNROLL_TRANSFORM -Wall -o sha2 sha2prog.c sha2.c
  % ./sha2test.pl

  [most of the perl script output deleted for brevity]

  ===== RESULTS (20 VECTOR DATA FILES HASHED) =====

  HASH TYPE       NO. OF TESTS    PASSED  FAILED
  ---------       ------------    ------  ------
  SHA-224                   20        20       0
  SHA-256                   20        20       0
  SHA-384                   20        20       0
  SHA-512                   20        20       0
  ----------------------------------------------
  TOTAL:                    80        80       0

  NO ERRORS!  ALL TESTS WERE SUCCESSFUL!

  ALL TEST VECTORS PASSED!

That's all folks!  Have fun!

Aaron out.