Friday, April 26, 2013

Gzip and Base64 in Dart

‹prev | My Chain | next›

To convert the ICE Code Editor to Dart, I will need the ability to read and write base64, gzip'd strings. The ICE Code editor gzips and base64 encodes code before storing it to localStorage. Unfortunately, this is almost certainly not going to work in Dart (at least not currently) because Dart's ZLibDeflater and ZLibInflater are part of the "dart:io" package, which is not available in the browser (noooooooooooo!!!!). Even so, I would like to give it a go just to see if it is capable of working with the equivalent JavaScript data.

The JavaScript files use js-deflate for the gzip compression/decompression and the built-in window.btoa() and window.atob() methods for base64 encoding. The result is that a simple code sample like:
<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>
Is stored in ICE as:
s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==
My first thought it to try to decode that in Dart. I import all of the necessary packages, declare the encoded code as str and then build up streams:
import 'dart:async';
import 'dart:io';
import 'dart:crypto';

main() {
  var str = "s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==";

  var controller = new StreamController();
  controller.stream
    .transform(new ZLibInflater())
    .fold([], (buffer, data) {
      print(data);
      buffer.addAll(data);
      return buffer;
    })
    .then((inflated) {
      print(new String.fromCharCodes(inflated));
      print(inflated);
    });
  controller.add(CryptoUtils.base64StringToBytes(str));
  controller.close();
}
I create a vanilla stream controller to which I can add data. The data that I add is the base64 decoded version of str. In the stream, I transform the data with a ZlibInflater instance, fold all of the data into a single buffer and then print out the resulting string. I think that ought to work, but...
➜  ice-code-editor git:(master) ✗ dart test.dart
s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==
[179, 73, 202, 79, 169, 180, 179, 209, 7, 83, 92, 54, 197, 201, 69, 153, 5, 37, 10, 197, 69, 201, 182, 74, 25, 37, 37, 5, 86, 250, 250, 233, 137, 185, 153, 121, 233, 94, 193, 122, 201, 249, 185, 250, 33, 25, 69, 169, 169, 122, 89, 197, 74, 64, 45, 16, 181, 68, 104, 114, 206, 40, 202, 207, 77, 117, 203, 172, 72, 45, 198, 174, 213, 142, 75, 65, 65, 95, 95, 33, 50, 191, 180, 72, 33, 57, 63, 37, 85, 33, 61, 63, 181, 88, 33, 35, 181, 40, 85, 79, 79, 143, 11, 174, 28, 0]
Uncaught Error: InternalError: 'Filter error, bad data'
Unhandled exception:
InternalError: 'Filter error, bad data'
#0      _FutureImpl._scheduleUnhandledError.<anonymous closure> (dart:async:325:9)
#1      Timer.run.<anonymous closure> (dart:async:2251:21)
#2      Timer.run.<anonymous closure> (dart:async:2259:13)
#3      Timer.Timer.<anonymous closure> (dart:async-patch:15:15)
#4      _Timer._createTimerHandler._handleTimeout (dart:io:6697:28)
#5      _Timer._createTimerHandler._handleTimeout (dart:io:6705:7)
#6      _Timer._createTimerHandler.<anonymous closure> (dart:io:6713:23)
#7      _ReceivePortImpl._handleMessage (dart:isolate-patch:81:92)
Ick.

I take an alternate approach to try to compress the same string to see what data comes back:
import 'dart:async';
import 'dart:io';
import 'dart:crypto';

main() {
  var str = "s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==";

  var code = '''<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>''';

  var controller = new StreamController();
  controller.stream
    .transform(new ZLibDeflater())
    .fold([], (buffer, data) {
      print(data);
      buffer.addAll(data);
      return buffer;
    })
    .then((inflated) {
      print(CryptoUtils.bytesToBase64(inflated));
      print(inflated);
    });
  controller.add(code.codeUnits);
  controller.close();
}
This results in a base64 string of:
H4sIAAAAAAAAA7JJyk+ptLPRB1NcNsXJRZkFJQrFRcm2ShklJQVW+vrpibmZeelewXrJ+bn6IRlFqal6WcVKQC0QtURocs4oys9NdcusSC3GrtWOS0FBX18hMr+0SCE5PyVVIT0/tVghI7UoVU9PjwuuHAAAAP//
Which is larger than what the JavaScript version has created:
s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==
I am at a loss to explain the difference, but the Dart version seems legit—I can even use this on the command line:
➜  ice-code-editor git:(master) ✗ echo -n "H4sIAAAAAAAAA7JJyk+ptLPRB1NcNsXJRZkFJQrFRcm2ShklJQVW+vrpibmZeelewXrJ+bn6IRlFqal6WcVKQC0QtURocs4oys9NdcusSC3GrtWOS0FBX18hMr+0SCE5PyVVIT0/tVghI7UoVU9PjwuuHAAAAP//" | base64 -d | gunzip -dc
<body></body>
<script src="http://gamingJS.com/Three.js"></script>
<script src="http://gamingJS.com/ChromeFixes.js"></script>
<script>
  // Your code goes here...
</script>
gzip: stdin: unexpected end of file
While the JavaScript version will not work:
➜  ice-code-editor git:(master) ✗ echo -n "s0nKT6m0s9EHU1w2xclFmQUlCsVFybZKGSUlBVb6+umJuZl56V7Besn5ufohGUWpqXpZxUpALRC1RGhyzijKz011y6xILcau1Y5LQUFfXyEyv7RIITk/JVUhPT+1WCEjtShVT0+PC64cAA==" | base64 -d | gunzip -dc

gzip: stdin: not in gzip format
It seems unlikely that I will be able to figure out why the two produce such different results. Sadly, it hardly matters. Since the Dart zlib classes are only available in "dart:io", which is only in the server-side VM, not the browser VM, I cannot use it anyway to store and retrieve data from localStorage. In the end, I will likely use js-interop to continue using the JavaScript version.


Day #733

No comments:

Post a Comment