LZ-based Compression Algorithm
Helper Script for Salesforce Commerce Cloud
If you are looking for solutions to work around the platform limits for Session Storage, here is a Helper Script for Salesforce Commerce Cloud (SFCC) to compress and decompress Objects and Strings.
This uses LZ-String to work around SFCC Character Limits by using Compression, as mentioned in this Trailhead Module.
SFCC Session Limits:
Strings are limited to 2000 characters
Session is limited to 4000 characters in total size
Helper
Create a new file named lzStringHelper.js and place it into your client cartridge scripts helper folder cartridge/scripts/helpers/lzStringHelper.js within your project.
Usage
Compressing UTF-8 Strings and Objects
Here is an example of how to use the helper file to compress a string.
// Require LZString Helper
var LZString = require('*/cartridge/scripts/helpers/lzStringHelper');
// Fake SFCC Object
var obj = {
zipcode: '33705',
storeId: '0083',
storeName: 'Store Name',
orderNo: '10000123',
paymentMethod: 'CREDIT_CARD'
};
// Fake String
var str = 'Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum.';
// Save Object to Session ( this will automatially be converted to a string )
session.custom.myCompressedObject = LZString.compress(obj);
// Save String to Session
session.custom.myCompressedString = LZString.compress(str);
Compression Results:
session.custom.myCompressedObject
㞂⁞ॠฌ惶İ⦈ׂ̰惬`⬈Ð耳耮瀄鈉ꈮ렁솱晓䁲Șୢ鴀攮䠀ᇴᇚ눪?〈쳙銀䳬䥅큀㭲Ť鎐Ũ醀愀䨁䐀諐ʠ쁁宎䀅
Decompressed: 112 Bytes
Compressed: 56 Bytes
Reduction: 50%
session.custom.myCompressedString
ಇ끎똀䂖p㎁嵠ጐۜ㉜ű聃⠠‚᠆㬤⚠ӵᎉ櫣ꁳ᠐눒ౌ䋠ꉅЄ限ዣ먔璈ꈥ颀⍰ႅᓃ鈌⣅淪≄䁇ᓄǐ쀊ꑂᴸ놈䥁쀁룹슑医烁킁‑肠䡀x䆂ထ쏐삡揩䋐쏩ᦂ웻㢢℉ꔋ?䏉䫐ぃ㫧렀褵ₒ쌦定꧐쉀≀Xﮢ易쇎蝠ꀠ蜈汁∰퇃ᖢ䭣ᤈꃀƘꏲ䔑턗⌑囥뎸ń틔ࠞ숆쏁㔘蓅죑癰琮償ⱀ⢟┏֢⇰酘澌أĀ㰞灞㶍棆䭨곘ᄽ埋Ņऒ㠍ૢ
Decompressed: 447 Bytes
Compressed: 160 Bytes
Reduction: 64%
Decompressing UTF-8 Strings and Objects
Here is an example of how to take the compressed string we created in the example above and decompress it.
// Require LZString Helper
var LZString = require('*/cartridge/scripts/helpers/lzStringHelper');
// Decompress Session Object
var obj = LZString.decompress(session.custom.myCompressedObject);
// Decompress Session String
var str = LZString.decompress(session.custom.myCompressedString);
Working with UTF-16
If you need to work with Base64 characters, use the following ( NOTE: This Base64 Compression is 166% larger than those produced by UTF-8 compress, but can still significantly compress some JSON objects. ):
// Save Object to Session
session.custom.myCompressedObject = LZString.compressToBase64(obj);
// Save String to Session
session.custom.myCompressedString = LZString.compressToBase64(str);
// Decompress Session Object
var obj = LZString.decompressFromBase64(session.custom.myCompressedObject);
// Decompress Session String
var str = LZString.decompressFromBase64(session.custom.myCompressedString);
Working with Base64
If you need to work with Base64 characters, use the following ( NOTE: This Base64 Compression is 166% larger than those produced by UTF-8 compress, but can still significantly compress some JSON objects. ):
// Save Object to Session
session.custom.myCompressedObject = LZString.compressToBase64(obj);
// Save String to Session
session.custom.myCompressedString = LZString.compressToBase64(str);
// Decompress Session Object
var obj = LZString.decompressFromBase64(session.custom.myCompressedObject);
// Decompress Session String
var str = LZString.decompressFromBase64(session.custom.myCompressedString);
Working with Encoded URI Components
If you need to work with Encoded URI characters, use the following ( NOTE: This Encoded URI Compression is similar to Base64 and 166% larger than those produced by UTF-8 compress, but can still significantly compress some JSON objects. ):
// Save Object to Session
session.custom.myCompressedObject = LZString.compressToEncodedURIComponent(obj);
// Save String to Session
session.custom.myCompressedString = LZString.compressToEncodedURIComponent(str);
// Decompress Session Object
var obj = LZString.decompressFromEncodedURIComponent(session.custom.myCompressedObject);
// Decompress Session String
var str = LZString.decompressFromEncodedURIComponent(session.custom.myCompressedString);
Working with Uint8Array
If you need to work with Uint8Array, use the following:
// Save Object to Session
session.custom.myCompressedObject = LZString.compressToUint8Array(obj);
// Save String to Session
session.custom.myCompressedString = LZString.compressToUint8Array(str);
// Decompress Session Object
var obj = LZString.decompressFromUint8Array(session.custom.myCompressedObject);
// Decompress Session String
var str = LZString.decompressFromUint8Array(session.custom.myCompressedString);
lzStringHelper.js
Here is the source code for cartridge/scripts/helpers/lzStringHelper.js
/**
* LZ-based Compression Algorithm
* This is a modified version of the following URL for use within SFCC
* @see https://github.com/pieroxy/lz-string/blob/master/libs/lz-string.js
*/
module.exports = (function() {
var f = String.fromCharCode;
var keyStrBase64 = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/=';
var keyStrUriSafe = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+-$';
var baseReverseDic = {};
/**
* Core Compress Function
*
* @param {Mixed} uncompressed String or Object to Compress
* @param {Integer} bitsPerChar Bits per Character
* @param {Integer} getCharFromInt Get Character from Integer
* @returns {String}
*/
function compress(uncompressed, bitsPerChar, getCharFromInt) {
if (uncompressed == null) return '';
if (typeof uncompressed !== 'string') {
try {
uncompressed = JSON.stringify(uncompressed)
} catch (err) {
return err.name + ': ' + err.message;
}
}
var i, value,
contextDictionary = {},
contextDictionaryToCreate = {},
contextC = '',
contextWC = '',
contextW = '',
contextEnlargeIn = 2,
contextDictSize = 3,
contextNumBits = 2,
contextData = [],
contextDataVal = 0,
contextDataPosition = 0,
ii;
for (ii = 0; ii < uncompressed.length; ii += 1) {
contextC = uncompressed.charAt(ii);
if (!Object.prototype.hasOwnProperty.call(contextDictionary, contextC)) {
contextDictionary[contextC] = contextDictSize++;
contextDictionaryToCreate[contextC] = true;
}
contextWC = contextW + contextC;
if (Object.prototype.hasOwnProperty.call(contextDictionary, contextWC)) {
contextW = contextWC;
} else {
if (Object.prototype.hasOwnProperty.call(contextDictionaryToCreate, contextW)) {
if (contextW.charCodeAt(0) < 256) {
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
}
value = contextW.charCodeAt(0);
for (i = 0; i < 8; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
} else {
value = 1;
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | value;
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = 0;
}
value = contextW.charCodeAt(0);
for (i = 0; i < 16; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
}
contextEnlargeIn--;
if (contextEnlargeIn == 0) {
contextEnlargeIn = Math.pow(2, contextNumBits);
contextNumBits++;
}
delete contextDictionaryToCreate[contextW];
} else {
value = contextDictionary[contextW];
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
}
contextEnlargeIn--;
if (contextEnlargeIn == 0) {
contextEnlargeIn = Math.pow(2, contextNumBits);
contextNumBits++;
}
contextDictionary[contextWC] = contextDictSize++;
contextW = String(contextC);
}
}
if (contextW !== '') {
if (Object.prototype.hasOwnProperty.call(contextDictionaryToCreate, contextW)) {
if (contextW.charCodeAt(0) < 256) {
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
}
value = contextW.charCodeAt(0);
for (i = 0; i < 8; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
} else {
value = 1;
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | value;
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = 0;
}
value = contextW.charCodeAt(0);
for (i = 0; i < 16; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
}
contextEnlargeIn--;
if (contextEnlargeIn == 0) {
contextEnlargeIn = Math.pow(2, contextNumBits);
contextNumBits++;
}
delete contextDictionaryToCreate[contextW];
} else {
value = contextDictionary[contextW];
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
}
contextEnlargeIn--;
if (contextEnlargeIn == 0) {
contextEnlargeIn = Math.pow(2, contextNumBits);
contextNumBits++;
}
}
value = 2;
for (i = 0; i < contextNumBits; i++) {
contextDataVal = (contextDataVal << 1) | (value & 1);
if (contextDataPosition == bitsPerChar - 1) {
contextDataPosition = 0;
contextData.push(getCharFromInt(contextDataVal));
contextDataVal = 0;
} else {
contextDataPosition++;
}
value = value >> 1;
}
while (true) {
contextDataVal = (contextDataVal << 1);
if (contextDataPosition == bitsPerChar - 1) {
contextData.push(getCharFromInt(contextDataVal));
break;
} else contextDataPosition++;
}
return contextData.join('');
}
/**
* Core Decompress Function
*
* @param {Integer} length String length
* @param {Integer} resetValue Reset Value
* @param {Function} getNextValue Callback Handler
* @returns {Mixed} Uncompressed String or Object
*/
function decompress(length, resetValue, getNextValue) {
var dictionary = [],
next,
enlargeIn = 4,
dictSize = 4,
numBits = 3,
entry = '',
result = [],
i,
w,
bits, resB, maxPower, power,
c,
data = {
val: getNextValue(0),
position: resetValue,
index: 1
};
for (i = 0; i < 3; i += 1) {
dictionary[i] = i;
}
bits = 0;
maxPower = Math.pow(2, 2);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
switch (next = bits) {
case 0:
bits = 0;
maxPower = Math.pow(2, 8);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
c = f(bits);
break;
case 1:
bits = 0;
maxPower = Math.pow(2, 16);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
c = f(bits);
break;
case 2:
return '';
}
dictionary[3] = c;
w = c;
result.push(c);
while (true) {
if (data.index > length) {
return '';
}
bits = 0;
maxPower = Math.pow(2, numBits);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
switch (c = bits) {
case 0:
bits = 0;
maxPower = Math.pow(2, 8);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
dictionary[dictSize++] = f(bits);
c = dictSize - 1;
enlargeIn--;
break;
case 1:
bits = 0;
maxPower = Math.pow(2, 16);
power = 1;
while (power != maxPower) {
resB = data.val & data.position;
data.position >>= 1;
if (data.position == 0) {
data.position = resetValue;
data.val = getNextValue(data.index++);
}
bits |= (resB > 0 ? 1 : 0) * power;
power <<= 1;
}
dictionary[dictSize++] = f(bits);
c = dictSize - 1;
enlargeIn--;
break;
case 2:
var output = result.join('');
try {
return JSON.parse(output)
} catch (err) {
return output
}
}
if (enlargeIn == 0) {
enlargeIn = Math.pow(2, numBits);
numBits++;
}
if (dictionary[c]) {
entry = dictionary[c];
} else {
if (c === dictSize) {
entry = w + w.charAt(0);
} else {
return null;
}
}
result.push(entry);
dictionary[dictSize++] = w + entry.charAt(0);
enlargeIn--;
w = entry;
if (enlargeIn == 0) {
enlargeIn = Math.pow(2, numBits);
numBits++;
}
}
}
/**
* Get Base Dictionary Value
*
* @param {String} alphabet Alphabet to use for Compression
* @param {String} character Current Character
* @returns {String} Dictionary Character
*/
function getBaseValue(alphabet, character) {
if (!baseReverseDic[alphabet]) {
baseReverseDic[alphabet] = {};
for (var i = 0; i < alphabet.length; i++) {
baseReverseDic[alphabet][alphabet.charAt(i)] = i;
}
}
return baseReverseDic[alphabet][character];
}
return {
/**
* Compress UTF-8 Strings
* Can be decompressed with `decompress`
* @param {Mixed} uncompressed String or Object to Compress
* @returns {String}
*/
compress: function(uncompressed) {
return compress(uncompressed, 16, function(a) {
return f(a);
});
},
/**
* Compress to Base64
*
* Produces ASCII UTF-16 strings representing the original string encoded in Base64.
* Can be decompressed with `decompressFromBase64`
* This works by using only 6bits of storage per character.
* The strings produced are therefore 166% bigger than those produced by compress.
* It can still reduce significantly some JSON compressed objects.
*
* @param {Mixed} uncompressed String or Object to Compress
* @returns {String}
*/
compressToBase64: function(uncompressed) {
if (uncompressed == null) return '';
var res = compress(uncompressed, 6, function(a) {
return keyStrBase64.charAt(a);
});
switch (res.length % 4) {
default:
case 0:
return res;
case 1:
return res + '===';
case 2:
return res + '==';
case 3:
return res + '=';
}
},
/**
* Compress to Encoded URI Component
*
* Produces ASCII strings representing the original string encoded in Base64 with a few tweaks to make these URI safe.
* Hence, you can send them to the server without thinking about URL encoding them. This saves bandwidth and CPU.
* These strings can be decompressed with `decompressFromEncodedURIComponent`.
* See the bullet point above for considerations about size.
*
* @param {Mixed} uncompressed String or Object to Compress
* @returns {String}
*/
compressToEncodedURIComponent: function(uncompressed) {
if (uncompressed == null) return '';
return compress(uncompressed, 6, function(a) {
return keyStrUriSafe.charAt(a);
});
},
/**
* Compress to Uint8Array
*
* Produces an uint8Array. Can be decompressed with `decompressFromUint8Array`
* @param {Mixed} uncompressed String or Object to Compress
* @returns {Uint8Array}
*/
compressToUint8Array: function(uncompressed) {
var compressed = compress(uncompressed);
var buf = new Uint8Array(compressed.length * 2);
for (var i = 0, TotalLen = compressed.length; i < TotalLen; i++) {
var currentValue = compressed.charCodeAt(i);
buf[i * 2] = currentValue >>> 8;
buf[i * 2 + 1] = currentValue % 256;
}
return buf;
},
/**
* Compress to UTF16
*
* Produces "valid" UTF-16 strings in the sense that all browsers can store them safely.
* So they can be stored in localStorage on all browsers tested.
* Can be decompressed with `decompressFromUTF16`
* This works by using only 15bits of storage per character.
* The strings produced are therefore 6.66% bigger than those produced by `compress`
*
* @param {*} uncompressed String or Object to Compress
* @returns {String}
*/
compressToUTF16: function(uncompressed) {
if (uncompressed == null) return '';
return compress(uncompressed, 15, function(a) {
return f(a + 32);
}) + ' ';
},
/**
* Decompress output generated from `compress`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompress: function(compressed) {
if (compressed == null) return '';
if (compressed == '') return null;
return decompress(compressed.length, 32768, function(index) {
return compressed.charCodeAt(index);
});
},
/**
* Decompress output generated from `compressToBase64`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompressFromBase64: function(compressed) {
if (compressed == null) return '';
if (compressed == '') return null;
return decompress(compressed.length, 32, function(index) {
return getBaseValue(keyStrBase64, compressed.charAt(index));
});
},
/**
* Decompress output generated from `compressToEncodedURIComponent`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompressFromEncodedURIComponent: function(compressed) {
if (compressed == null) return '';
if (compressed == '') return null;
compressed = compressed.replace(/ /g, '+');
return decompress(compressed.length, 32, function(index) {
return getBaseValue(keyStrUriSafe, compressed.charAt(index));
});
},
/**
* Decompress output generated from `compressToUint8Array`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompressFromUint8Array: function(compressed) {
if (compressed === null || compressed === undefined) {
return decompress(compressed);
} else {
var buf = new Array(compressed.length / 2);
for (var i = 0, TotalLen = buf.length; i < TotalLen; i++) {
buf[i] = compressed[i * 2] * 256 + compressed[i * 2 + 1];
}
var result = [];
buf.forEach(function(c) {
result.push(f(c));
});
return decompress(result.join(''));
}
},
/**
* Decompress output generated from `compressToUTF16`
*
* @param {String} compressed Compress String
* @returns {Mixed} Decompressed String or Object
*/
decompressFromUTF16: function(compressed) {
if (compressed == null) return '';
if (compressed == '') return null;
return decompress(compressed.length, 16384, function(index) {
return compressed.charCodeAt(index) - 32;
});
}
};
})();