Downloads & Manuals

Using hzip is easy enough that I won't have to make a .pdf file to describe it. Find the heading that describes your current need and below listed will be the command to make it happen with hzip.

Downloading and Requirements
There are a two options:
  1. You can obtain the jar file here. This file contains all .jar files that you will need to invoke hzip using java. The command line for hzip will be: `java -jar HzipApp.jar:hadoop_conf_dir org.yagnus.hzip.HZip ...`
  2. You can build hzip from its source code. This will require you to get the scala compiler 2.8.0 or newer from here. You will need to be running Apache hadoop 0.20 or newer. Additionally Apache Compress is being used to create and decompress bzip2 files, so that needs to be in the build classpath as well.

For the remainder of this documentation, the text between the quotes will be shortened to hzip.

Decompressing ".gz" files
andrew%> hadoop dfs -dus /data/*
hdfs://andrew.yagn.us:54321/data/a.gz 897598375
hdfs://andrew.yagn.us:54321/data/b.gz 98759878
andrew%> hzip gunzip /data/a.gz /data/b.gz
andrew%> hadoop dfs -dus /data/*
hdfs://andrew.yagn.us:54321/data/a.gz 897598375
hdfs://andrew.yagn.us:54321/data/b.gz  98759878
hdfs://andrew.yagn.us:54321/data/a    897598375
hdfs://andrew.yagn.us:54321/data/b    128759878
andrew%>

Decompressing ".bz" files
andrew%> hadoop dfs -dus /data/*
hdfs://andrew.yagn.us:54321/data/a.bz2 897598375
hdfs://andrew.yagn.us:54321/data/b.bz2 98759878
andrew%> hzip bunzip /data/a.bz2 /data/b.bz2
andrew%> hadoop dfs -dus /data/*
hdfs://andrew.yagn.us:54321/data/a.bz2 897598375
hdfs://andrew.yagn.us:54321/data/b.bz2  98759878
hdfs://andrew.yagn.us:54321/data/a    947598375
hdfs://andrew.yagn.us:54321/data/b    158759878
andrew%>

Decompressing ".zip" files
andrew%> hadoop dfs -dus /data/*
hdfs://andrew.yagn.us:54321/data/a.zip 897598375
hdfs://andrew.yagn.us:54321/data/b.zip 98759878
andrew%> hzip unzip /data/a.zip /data/b.zip
andrew%> hadoop dfs -dus /data/*
hdfs://andrew.yagn.us:54321/data/a.zip 897598375
hdfs://andrew.yagn.us:54321/data/b.zip  98759878
hdfs://andrew.yagn.us:54321/data/a     877598375
hdfs://andrew.yagn.us:54321/data/b     118759878
andrew%>
Comments