$Id: hashall.txt,v 3.6 2023/12/30 11:30:34 ralph Exp $ CRLF, ASCII-DOC, (c) by Ralph Roth, ROSE_SWE

1. What is Hashall?

Hashall is a hash checking program for the command line, also known as a hash verifier or checksum calculator, a software tool that calculates and verifies the integrity of data files or messages. Hash checking programs are commonly used to verify the authenticity of downloaded files, ensure data integrity during file transfers, and detect data tampering or corruption.

A hash function is a mathematical algorithm that generates a fixed-size, unique digital fingerprint of the data being checked. Hash checking programs use this fingerprint, also known as a hash or checksum, to compare with the expected hash or checksum value to determine if the data has been altered or corrupted during transmission.

For example, when you download a file from the Internet, you can use hashall to calculate the file’s hash value and compare it to the original hash value provided by the file’s author. If the two values match, this indicates that the file has not been altered in transit and is the original file. If the values do not match, it may indicate that the file has been corrupted or tampered with, and you may need to download the file again.

Hashall calculates and prints out the most common hash types used.

Hashall is a multi-platform program, available for the most common platforms.

2. Common Hash Types

MD5, SHA-1, SHA-2, and SHA-3 are cryptographic hash functions (families), which are algorithms that generate a fixed-size, unique digital fingerprint of a message or data file. These hash functions are widely used in computer security, digital forensics, and data integrity verification:

  • MD5 (Message Digest 5) is a widely used hash function that generates a 128-bit hash value. It is no longer considered secure for cryptographic purposes due to vulnerabilities in the algorithm.

  • SHA-1 (Secure Hash Algorithm 1) is another widely used hash function that generates a 160-bit hash value. However, it has been found to be vulnerable to collision attacks, and it is no longer recommended for cryptographic use.

  • SHA-256 (Secure Hash Algorithm 256) is a newer and more secure hash function that generates a 256-bit hash value. It is widely used for digital signatures, data integrity checks, and other cryptographic applications.

  • SHA-3 (Secure Hash Algorithm 3) is a family of hash functions that were selected as the winners of the NIST hash function competition in 2012. The most commonly used SHA-3 variant generates a 256-bit hash value, but other variants are also available. SHA-3 is considered to be more secure and efficient than its predecessors.

Note
In summary, MD5 and SHA-1 are older hash functions that are no longer considered secure for cryptographic purposes. SHA-256 is currently widely used for cryptographic applications, while SHA-3 is a newer and more secure alternative that is gaining popularity.

2.1. KECCAK

Keccak is a family of cryptographic hash functions designed by Guido Bertoni, Joan Daemen, Michaël Peeters, and Gilles Van Assche. It was selected as the basis for the SHA-3 standard by the National Institute of Standards and Technology (NIST) in 2012, after a five-year competition that evaluated dozens of candidate hash functions.

Keccak is based on a sponge construction, which allows for flexible output lengths and message sizes. The algorithm processes the input message in a series of rounds, where each round applies a permutation function to the state of the sponge. The output of the hash function is derived from the final state of the sponge.

Keccak is designed to be highly secure, with a strong resistance to known attacks, such as collision attacks, preimage attacks, and birthday attacks. It is also highly versatile, with variable output lengths ranging from 224 bits to 512 bits, and supports various input message sizes. Keccak has become increasingly popular in recent years, and it is now used in various applications, including digital signatures, password hashing, key derivation, and other cryptographic protocols.

2.2. SHAKE

SHAKE128 and SHAKE256 are cryptographic hash functions that belong to the SHA-3 family of hash functions. It was created by Guido Bertoni, Joan Daemen, Michaël Peeters and Gilles Van Assche, and was selected as one of the five finalists in the NIST Hash Function Competition in 2012, before being standardised as part of the the SHA-3 standard. The name "SHAKE" derives from the fact that the hash function is based on a "squeezing" operation that produces an output of variable length. The "128" in the name indicates the size of the output, which can range from from 1 to 2^128 bits. The "256" in the name indicates the size of the output, which can range from 1 to 2^256 bits.

SHAKE is based on the Keccak sponge construction, which allows for flexible output lengths and message sizes. It is designed to be highly secure, with a strong resistance to known attacks, such as collision attacks, preimage attacks, and birthday attacks. It is also highly versatile, with support for variable-length outputs, making it suitable for a wide range of applications, including digital signatures, key derivation, and other cryptographic protocols.

In summary, SHAKE128 and SHAKE256 is a secure and versatile hash function that can produce variable length outputs of up to 2^128 or 2^256 bits. It is widely used in various cryptographic applications, especially those requiring a high degree of security and flexibility.

2.3. XXHash

Note
Requires the command line option "-xxhash". The XXHash algorithm used is limited to 2GB big files, so don’t use this on files greater than 2GB! Also the XXHash implementation allocates the same memory as the file to check is in size. Means don’t use this option on a small VM with 512MB of memory and a 1.5GB file.

XXHash is a non-cryptographic hash function that is designed for speed and efficiency. It was developed by Yann Collet and first released in 2012. XXHash is often used in applications where fast hash computation is critical, such as in big data processing, content-addressable storage, and data streaming.

XXHash is based on a hash function called "MurmurHash," but with several optimizations to improve speed and reduce collisions. It works by processing input data in 4-byte chunks, and applying a series of bitwise and arithmetic operations to produce a 32-bit hash value. One of the key advantages of XXHash is its speed. It can hash data much faster than many other hash functions, including popular cryptographic hash functions such as SHA-256 or SHA-3. This makes it ideal for use in high-performance applications where speed is a critical factor.

However, it’s important to note that XXHash is not a cryptographic hash function, and should not be used for applications that require strong security properties. XXHash is susceptible to collision attacks and is not designed to provide the same level of security as cryptographic hash functions like SHA-256 or SHA-3.

3. Usage

HASHALL -- (c) 2016-2024 by ROSE SWE, Ralph Roth, http://rose.rult.at/

usage: hashall [-short] [-time] file(s)

Prints various MD5, SHA1, SHA2, SHA3 hashes (Keccak) for the file(s) passed
on the command line argument.   SHA3_256 => SHA3 commonly used
Keccak_b, SHAKE128 and SHAKE256 uses a variable hash length,
we use 256 bits here.

Hashall reads the file for each hash from disk and is for this reason very
slow on big files, but on the other hand does not need to allocate e.g.
for a Double-DVD 9 GB of memory :-)

-short       Short, skips SHA2 and SHA3 family output
-time        Print the time needed for hash calculating [mm:ss]
-xxhash      Calculate XXHash128 (file must be smaller than 2GB)

$Id: hashall.txt,v 3.6 2023/12/30 11:30:34 ralph Exp $ - Freeware!

4. Example

$ hashall -time de_office_professional_plus_2021_x86_x64_dvd_3877611d.iso
----=[ de_office_professional_plus_2021_x86_x64_dvd_3877611d.iso ]=------------------
Filesize                4.420.782.080  (0x01077FC800) hex
[00:14]  MD5            B1B0BD8643E154AAE6797CF181C29CB2
[00:25]  SHA1 (sha160)  9B57E626A0D59CA65A664C98E0029AB417D52ED4
---=[ SHA2 Family ]=---
[00:50]  sha224         20A9879D14976E0A7BA547F8E6EFE9C4492666E8430BBCBEAD3E7E63
[00:50]  sha256         5837E03AF67CFBBD938948918DD1BD0E46B1E873060E668C716C1EE91E757BBB
[00:30]  sha384         E0B14A39DE333ED532C08B51454226FED0D34D4624D561455321B3C4EFAC68B1E6052631E087956C81DF2AB70AC50495
[00:30]  sha512         36C2CE7D69B40DDBC591178EBF69DA83182A9A3A025F8E73C2EB423881B3AF3748E208DACC7FD1952FEAE251B8CE61FE1E5A866BAB4237A7D371EA6FBFCFFC91
[00:30]  sha512_224     0C1B29C376D4331F254BF23E34C5499199310B5CFC57E7502863417C
[00:30]  sha512_256     56936BD7A429FDBF270628C7CE3EBCD32309C43600564724BE684DB08CCB7DA3
---=[ SHA3 Family ]=---
[02:54]  Keccak224      291D27F362A476BA26C3D5581AE13358E0FDBBD861B562E781EB456B
[03:05]  Keccak256      71687473110C932D4CAE683C736FBAF9BA29759B17319EF19FBE52DD1BF63A66
[04:03]  Keccak384      C2BDAEB779965E9D27FD24EB8084464761E9C5A5FBCCA3D0815460E2821BCE8C09901212BE690C369D1E91C3174384AA
[05:50]  Keccak512      670680157A6DAAB9E288347B4D7AF4F39E2106E0305490189041008FB283F7D661BDF9F7A78A18C1BE828DC9E894C47ECE9525E2DF76CBD9B290E2799F6B16A5
[03:18]  Keccak_b       5A38EB71C23914173FA3358A3AB7BEA0E55C548509CAE7592BB1D03D45AF033F
[02:56]  SHA3_224       1737DCD42EADB8659AED37035DD931772DCCC1EBA271EF75A3F475B1
[03:06]  SHA3_256       B4BAFC50FB40A74FA7E079FCBA01BD3C04C5C10647C56825D06C111E781AADB0
[04:16]  SHA3_384       42061B58956AE1BFE4C07F8996F35A13D6175C75A4CD8637162F43ACC73A20B768021BCEA35F48E1EF4C8936D030CD2B
[05:49]  SHA3_512       04AB6CCA9662C1290C7B698FB99AE27B6372F38052048DF69D278AC11A5432BB71B79AC17E5C7680D614FC111C4CAAA13594D4A3FB7F77AB865124962DF2B601
[02:30]  SHAKE128       20CE43EB49B06AF40A3BA91B61172B83AD24863FAB5B90F78CD863EE25355F54
[03:06]  SHAKE256       E34204B1DD00746BDBEE9181AB2F80560636558275BBD078B78A7D82D517BDB3
XXHash
$ time xxhsum  de_office_professional_plus_2021_x86_x64_dvd_3877611d.iso
f9f616aad47d6b96  de_office_professional_plus_2021_x86_x64_dvd_3877611d.iso
real 0,665	user 0,321	sys 0,344	pcpu 99,99
Note
SHA3_512 and Keccak512 are at least factor 10-20 slower that e.g. MD5

/* end of document */