Generate Unique Resource IDs with Checksum in Node.js Using Base32 Encoding

Introduction

When working with Node.js, generating unique resource IDs with base32 encoding and checksum verification ensures data integrity and security. This process involves creating identifiers from random bytes, encoding them using base32, and appending a checksum to verify their accuracy and prevent tampering. Whether you’re building APIs or managing databases, these techniques help ensure that each ID remains unique, readable, and permanent. In this tutorial, we’ll walk through the steps to create and validate these resource IDs using Node.js, base32 encoding, and checksum calculations.

Step 1 — Generating an Encoded ID

In this step, you will write a function to generate an identifier from random bytes into a unique alphanumeric string. Your identifier will be encoded using base32 encoding, but it will not have a checksum affiliated until later in the tutorial. The encoding process will create a unique identifier of a specified length based on the number of bytes you choose, building an ID that incorporates some of the characteristics of a good ID.

Start by making a new folder for this project, then move into that folder:

$ mkdir checksum

$ cd checksum

The project folder will be called checksum for this tutorial. Create and open a package.json file in your project folder (using your favorite editor):

$ nano package.json

Then add the following lines of code:

{
  “name”: “checksum”,
  “version”: “1.0.0”,
  “main”: “index.js”,
  “type”: “module”
}

In this file, you define the project name as checksum, and you consider the code version “1.0.0”. You define the main JavaScript file as index.js. When you have “type”: “module” in the package.json file, your source code should use import syntax. In this file, you use the JSON data format, which you can learn more about in How to Work with JSON in JavaScript. Save and close the file.

You’ll use a few Node.js modules to generate the ID: crypto and base32-encode, with its corresponding decoder base32-decode. The crypto module is packaged with Node.JS, but you will need to install base32-encode and base32-decode for use later in this tutorial. Encoding is putting a sequence of characters (letters, numbers, punctuation, and certain symbols) into a specialized format for efficient transmission or storage. Decoding is the opposite process: converting an encoded format back into the original sequence of characters. Base32 encoding uses a 32-character set, which makes it a textual 32-symbol notation for expressing numbers.

In a terminal session, install these module packages in the project folder with the following command:

$ npm i base32-encode base32-decode

You will receive an output that indicates these modules have been added:

added 3 packages, and audited 5 packages in 2s
found 0 vulnerabilities

If you encounter issues during installation, you can refer to How To Use Node.js Modules with npm and package.json for support. Still in your project folder, create a new file called index.js:

$ nano index.js

Add the following lines of JavaScript code to the index.js file:

import crypto from ‘crypto’;
import base32Encode from ‘base32-encode’;
import base32Decode from ‘base32-decode’;</p>
<p>function generate_Id(byte_size) {
  const bytes = crypto.randomBytes(byte_size);
  return base32Encode(bytes, ‘Crockford’);
}</p>
<p>console.log(‘ID for byte size = 1:’, generate_Id(1), ‘\n’);
console.log(‘ID for byte size = 12:’, generate_Id(12), ‘\n’);
console.log(‘ID for byte size = 123:’, generate_Id(123), ‘\n’);

The import command loads the required modules. To generate bytes from the number, you define a generate_Id function to take the bytes’ size of bytes and then create random bytes of this size using the randomBytes function from the crypto module. The generate_Id function then encodes these bytes using the Crockford implementation of base32 encoding. For instructional purposes, a few IDs are generated and then logged to the console. The base32-decode module will be used to decode the resource ID in the next steps.

Save your index.js file, then run the code in a terminal session with this command:

$ node index.js

You will receive an output response similar to this:

ID for byte size = 1: Y8
ID for byte size = 12: JTGSEMQH2YZFD3H35HJ0
ID for byte size = 123: QW2E2KJKM8QZ7174DDB1Q3JMEKV7328EE8T79V1KG0TEAE67DEGG1XS4AR57FPCYTS24J0ZRR3E6TKM28AM8FYZ2AZTZ55C9VVQTABE0R7QRH7QBY7V3GBYBNN5D9JK0QMD9NXSWZN95S0772DHN43Q003G0QNTPA2J3AFA3P7Q167C1VNR92Z85PCDXCMEY0M7WA

Your ID values might differ due to the randomness of generated bytes. The generated ID may be shorter or longer in length, depending on the byte size you select. Back in index.js, comment out the console outputs using the JavaScript commenting feature (adding a double slash // before the line):

//console.log(‘ID for byte size = 1:’, generate_Id(1), ‘\n’);
//console.log(‘ID for byte size = 12:’, generate_Id(12), ‘\n’);
//console.log(‘ID for byte size = 123:’, generate_Id(123), ‘\n’);

These lines demonstrate how encoding will output different identifiers based on the bytes associated. Because these lines will not be used in the following sections, you can comment them out as demonstrated in this code block or delete them entirely. In this step, you created an encoded ID by encoding random bytes. In the next step, you will combine the encoded bytes and a checksum, creating a unique identifier.

Step 2 — Generating a Resource Identifier

Now you will create an ID with a checksum character. Generating the checksum character is a two-step process. For instructional purposes, each function that creates the composite function will be built separately in the following subsections. First, you will write a function that runs a modulo operation. Then, you will write another function that maps the results to a checksum character, which is how you will generate the checksum for your resource ID. Finally, you will verify the identifier and checksum to ensure that the resource identifier is accurate.

Step 3 — Running a Modulo Operation

In this section, you will convert the bytes corresponding to the number ID to a number between 0-36 (limits inclusive, which means any number between 0 to 36, including 0 and 36). The bytes corresponding to the number ID are converted to an integer as a result of a modulo operation. The modulo operation will return the remainder of the dividend obtained by converting the bytes to BigInteger (BigInt) values. To implement this procedure, add the following lines of code to the bottom of the index.js file:

function calculate_checksum(bytes) {
  const intValue = BigInt(`0x${bytes.toString(‘hex’)}`);
  return Number(intValue % BigInt(37));
}

The function calculate_checksum works with the bytes defined earlier in the file. This function will convert bytes to hexadecimal values, which are further converted to BigInteger BigInt values. The BigInt data type represents numbers greater than those represented by the primitive data type number in Javascript. For example, although integer 37 is relatively small, it is converted to BigInt for the modulo operation. To achieve this conversion, you first set the intValue variable with the BigInt conversion method, using the toString method to set bytes to hex. Then, you return a numerical value with the Number constructor, in which you run the modulo operation with the % symbol to find the remainder between the intValue and BigInt using the sample value of 37. That integer value (in this example, 37) acts as an index to select an alphanumeric character from a custom-built string of alphanumeric characters. If intValue value is 123 (depending on the bytes), the module operation will be 123 % 37. The result of this operation with 37 as the integer value will be a remainder of 12 and a quotient of 3. With a value of 154 for the resource ID, the operation 154 % 37 will result in a remainder of 6. This function maps the incoming bytes to the modulo result.

Step 4 — Obtaining a Checksum Character

After obtaining the modulo result in the previous section, you can map it to a checksum character. Add the following lines of code to the index.js file just below the previous code:

function get_checksum_character(checksumValue) {
  const alphabet = ‘0123456789ABCDEFG’ + ‘HJKMNPQRSTVWXYZ*~$=U’;
  return alphabet[Math.abs(checksumValue)];
}

For the function get_checksum_character, you call checksumValue as a parameter. Within this function, you define a string constant named alphabet as a custom-built string of alphanumeric characters. Depending on the value set for the checksumValue, this function will return a value that pairs the defined string from the alphabet constant with the absolute value of the checksumValue.

Step 5 — Generating ID with Checksum

Next, you will write a function that uses the two functions written in these sections to generate an ID from the encoding of bytes combined with a checksum character. Add the following lines of code to the index.js file:

function generate_Id_with_checksum(bytes_size) {
  const bytes = crypto.randomBytes(bytes_size);
  const checksum = calculate_checksum(bytes);
  const checksumChar = get_checksum_character(checksum);
  console.log(“checksum character: “, checksumChar);
  const encoded = base32Encode(bytes, ‘Crockford’);
  return encoded + checksumChar;
}</p>
<p>const Hotel_resource_id = generate_Id_with_checksum(132);
console.log(“Hotel resource id: “, Hotel_resource_id);

Step 6 — Verifying the ID

The verify_Id function checks the integrity of the ID by checking the checksum. The remaining characters of the identifier are decoded into a buffer, and then calculate_checksum and get_checksum_character are run subsequently on this buffer to extract the checksum character for the comparison (with calculated_checksum_char == checksum_char).

Step 7 — Altering the Identifier for a Non-matching Result

You will now alter the value for the identifier to check if the checksums will get matched. The alteration in this step will always result in a non-matching checksum, as the integrity is not maintained if any character in the ID is manipulated. An alteration like this may result from transmission errors or malicious behavior. This alteration is for instructional purposes and is not recommended for production builds but will enable you to assess a non-matching checksum result.

You can learn more about generating unique identifiers and checksums in programming through this detailed guide on Node.js Resource ID Generation and Checksum Techniques.

Conclusion

In conclusion, generating unique resource IDs with a checksum in Node.js using base32 encoding is a powerful method for ensuring data integrity and security. By creating an identifier from random bytes, encoding it, and appending a checksum for verification, you can guarantee that the ID remains unaltered and reliable. This process highlights key qualities like uniqueness, readability, and permanence, which are essential for any robust API or data management system. As you apply these techniques, remember that leveraging base32 encoding and checksum verification is crucial in maintaining secure, tamper-proof identifiers in your applications. Looking ahead, these practices will continue to evolve as the demand for secure and scalable systems grows, particularly in API development and database management.

Nodemon: How to Auto-Restart Node.js Apps on Save