Update documentation

Signed-off-by: Chris Lalancette <[email protected]>

custom-pydoc.py

@@ -1,6 +1,9 @@
-#!/usr/bin/python2.7
+#!/usr/bin/python3
 
-import StringIO
+try:
+    from cStringIO import StringIO as BytesIO
+except ImportError:
+    from io import BytesIO
 
 import pydoc
 import lxml.etree
@@ -13,7 +16,7 @@
 
 # Now parse that documentation
 parser = lxml.etree.HTMLParser()
-tree = lxml.etree.parse(StringIO.StringIO(page), parser)
+tree = lxml.etree.parse(BytesIO(page.encode('ascii')), parser)
 
 # Now we remove the "Modules" section, since it contains only links to parts
 # of the API that we are not documenting
@@ -32,4 +35,4 @@
     remove_table.getparent().remove(remove_table)
 
 # Print out the results
-print(lxml.etree.tostring(doc))
+print(lxml.etree.tostring(doc).decode('ascii'))

docs/_config.yml

@@ -1 +1 @@
-theme: jekyll-theme-hacker
+theme: jekyll-theme-slate

docs/design.md

@@ -0,0 +1,16 @@
+# Design
+
+---
+
+<div style="width: 100%; display: table;">
+  <div style="display: table-row;">
+    <div style="width: 33%; display: table-cell; text-align: left;">
+      <a href="example-reading-file-in-chunks.html"><-- Example: Reading a large file in chunks</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: center;">
+      <a href="https://clalancette.github.io/pycdlib/">Top</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: right;">
+      <a href="tools.html">Tools --></a>
+    </div>
+</div>

docs/example-creating-bootable-iso.md

@@ -0,0 +1,82 @@
+# Example: Creating a bootable ISO (El Torito)
+
+This example will show how to create a bootable [El Torito](standards.md#el-torito) ISO.  Here's the complete code for the example:
+
+```
+try:
+    from cStringIO import StringIO as BytesIO
+except ImportError:
+    from io import BytesIO
+
+import pycdlib
+
+iso = pycdlib.PyCdlib()
+
+iso.new()
+
+bootstr = b'boot\n'
+iso.add_fp(BytesIO(bootstr), len(bootstr), '/BOOT.;1')
+
+iso.add_eltorito('/BOOT.;1')
+
+iso.write('eltorito.iso')
+
+iso.close()
+```
+
+Let's take a closer look at the code.
+
+```
+try:
+    from cStringIO import StringIO as BytesIO
+except ImportError:
+    from io import BytesIO
+
+import pycdlib
+```
+
+As usual, import the necessary libraries, including pycdlib.
+
+```
+iso = pycdlib.PyCdlib()
+
+iso.new()
+```
+
+Create a new PyCdlib object, and then create a new, basic ISO.
+
+```
+bootstr = b'boot\n'
+iso.add_fp(BytesIO(bootstr), len(bootstr), '/BOOT.;1')
+```
+
+Add a file called /BOOT.;1 to the ISO.  This is the file that contains the data to be used to boot the ISO when placed into a computer.  The name of the file can be anything (and can even be nested in directories), but the contents have to be very specific.  Getting the appropriate data into the boot file is beyond the scope of this tutorial; see [isolinux](http://www.syslinux.org/wiki/index.php?title=ISOLINUX) for one way of getting the appropriate data.  Suffice it to say that the example code that we are using above will not actually boot, but is good enough to show the PyCdlib API usage.
+
+```
+iso.add_eltorito('/BOOT.;1')
+```
+
+Add El Torito to the ISO, making the boot file "/BOOT.;1".  After this call, the ISO is actually bootable.  By default, the [add_eltorito](pycdlib-api.html#PyCdlib-add_eltorito) method will use so-called "no emulation" booting, which allows arbitrary data in the boot file.  "Hard drive" and "floppy" emulation is also supported, though these options are more esoteric and need specifically configured boot data to work properly.
+
+```
+iso.write('eltorito.iso')
+
+iso.close()
+```
+
+Write the ISO out to a file, and close out the PyCdlib object.
+
+---
+
+<div style="width: 100%; display: table;">
+  <div style="display: table-row;">
+    <div style="width: 33%; display: table-cell; text-align: left;">
+      <a href="example-extracting-data-from-iso.html"><-- Example: Extracting data from an existing ISO</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: center;">
+      <a href="https://clalancette.github.io/pycdlib/">Top</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: right;">
+      <a href="example-creating-rock-ridge-iso.html">Example: Creating an ISO with Rock Ridge --></a>
+    </div>
+</div>

docs/example-creating-hybrid-bootable-iso.md

@@ -0,0 +1,92 @@
+# Example: Creating a "hybrid" bootable ISO
+
+The first 32768 bytes of any ISO are designated as "system use".  In a normal ISO (even an El Torito bootable one), these bytes are all zero, but this space can also be used to add in alternative booting mechanisms.  In particular, this space can be used to embed boot code so that the file can be written to a USB stick and booted.  These so called "hybrid" ISO files thus have two booting mechanisms: if the file is actually burned to a CD, then "El Torito" is used to boot, but if it is written to a USB stick, then the system use boot code is used to boot.  PyCdlib supports creating hybrid bootable ISOs through the main API, and the following example will show how.
+
+Here's the complete code for the example:
+
+```
+try:
+    from cStringIO import StringIO as BytesIO
+except ImportError:
+    from io import BytesIO
+
+import pycdlib
+
+iso = pycdlib.PyCdlib()
+
+iso.new()
+
+bootstr = b'boot\n'
+iso.add_fp(BytesIO(bootstr), len(bootstr), '/BOOT.;1')
+
+iso.add_eltorito('/BOOT.;1')
+
+iso.add_isohybrid()
+
+iso.write('eltorito.iso')
+
+iso.close()
+```
+
+Let's take a closer look at the code.
+
+```
+try:
+    from cStringIO import StringIO as BytesIO
+except ImportError:
+    from io import BytesIO
+
+import pycdlib
+```
+
+As usual, import the necessary libraries, including pycdlib.
+
+```
+iso = pycdlib.PyCdlib()
+
+iso.new()
+```
+
+Create a new PyCdlib object, and then create a new, basic ISO.
+
+```
+isolinuxstr = b'\x00'*0x40 + b'\xfb\xc0\x78\x70'
+iso.add_fp(BytesIO(isolinuxstr), len(isolinuxstr), '/BOOT.;1')
+```
+
+Add a file called /BOOT.;1 to the ISO.  The contents of this conform to the expected boot start sequence as specified by isolinux.  A complete discussion of the correct form of the file is out of scope for this tutorial; see [isolinux](http://www.syslinux.org/wiki/index.php?title=ISOLINUX) for more details.  The above is the minimum code that conforms to the sequence, though it is not technically bootable.
+
+```
+iso.add_eltorito('/BOOT.;1', boot_load_size=4)
+```
+
+Add El Torito to the ISO, making the boot file "/BOOT.;1", and setting the `boot_load_size` to 4.  The `boot_load_size` is the number of 512-bytes sectors to read during initial boot.  While other values may be allowed for this, all current examples (from cdrkit or isolinux) use this value.  After this call, the ISO is El Torito bootable, but not yet a hybrid ISO.
+
+```
+iso.add_isohybrid()
+```
+
+Add the boot file to the system use area, making this a hybrid ISO.  There are various parameters that can be passed to control how the hybrid file is added, but the defaults are typically good enough for creating a hybrid ISO similar to those made for most Linux distributions.
+
+```
+iso.write('eltorito.iso')
+
+iso.close()
+```
+
+Write the ISO out to a file, and close out the PyCdlib object.
+
+---
+
+<div style="width: 100%; display: table;">
+  <div style="display: table-row;">
+    <div style="width: 33%; display: table-cell; text-align: left;">
+      <a href="example-forcing-consistency.html"><-- Example: Forcing consistency</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: center;">
+      <a href="https://clalancette.github.io/pycdlib/">Top</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: right;">
+      <a href="example-walking-iso-filesystem.html">Example: Walking the ISO filesystem --></a>
+    </div>
+</div>

docs/example-creating-joliet-iso.md

@@ -0,0 +1,75 @@
+# Example: Creating an ISO with Joliet extensions
+
+This example will show how to create an ISO with the [Joliet](standards.md#joliet) extensions.  Here's the complete code for the example:
+
+```
+try:
+    from cStringIO import StringIO as BytesIO
+except ImportError:
+    from io import BytesIO
+
+import pycdlib
+
+iso = pycdlib.PyCdlib()
+iso.new(joliet=3)
+foostr = b'foo\n'
+iso.add_fp(BytesIO(foostr), len(foostr), '/FOO.;1', joliet_path='/foo')
+iso.add_directory('/DIR1', joliet_path='/dir1')
+iso.write('new.iso')
+iso.close()
+```
+
+Let's take a closer look at the code.
+
+```
+try:
+    from cStringIO import StringIO as BytesIO
+except ImportError:
+    from io import BytesIO
+
+import pycdlib
+```
+
+As in earlier examples, import the relevant libraries, including pycdlib itself.
+
+```
+iso = pycdlib.PyCdlib()
+iso.new(joliet=3)
+```
+
+Create a new PyCdlib object, and then create a new ISO with that object.  In order to make it have Joliet extensions, we pass the argument `joliet=3` to the [new](pycdlib-api.html#PyCdlib-new) method.  PyCdlib supports Joliet levels 1, 2, and 3, but level 3 is by far the most common, so is recommended.
+
+```
+foostr = b'foo\n'
+iso.add_fp(BytesIO(foostr), len(foostr), '/FOO.;1', joliet_path='/foo')
+```
+
+As in earlier examples, create a new file on the ISO from a string.  Because this is a Joliet ISO, we have to provide the `joliet_path` argument to [add_fp](pycdlib-api.html#PyCdlib-add_fp) as well.  In contrast to Rock Ridge, Joliet is a completely different namespace from the original ISO9660 structure, and so the argument to be passed here must be an absolute path, not a name.  Because of this, the Joliet file can be on a completely different part of the directory structure, or be omitted completely (in which case the file will only show up on the ISO9660 portion of the ISO).  In practice the Joliet portion of the ISO almost always mirrors the ISO9660 portion of the ISO, so it is recommended to do that when creating new Joliet ISOs.
+
+```
+iso.add_directory('/DIR1', joliet_path='/dir1')
+```
+
+Create a new directory on the ISO.  Again we must pass the `joliet_path` argument to [add_directory](pycdlib-api.html#PyCdlib-add_directory), for all of the same reasons and with the same restrictions as we saw above for [add_fp](pycdlib-api.html#PyCdlib-add_fp).
+
+```
+iso.write('new.iso')
+iso.close()
+```
+
+Write the new ISO out to a file, then close out the ISO.
+
+---
+
+<div style="width: 100%; display: table;">
+  <div style="display: table-row;">
+    <div style="width: 33%; display: table-cell; text-align: left;">
+      <a href="example-creating-rock-ridge-iso.html"><-- Example: Creating an ISO with Rock Ridge</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: center;">
+      <a href="https://clalancette.github.io/pycdlib/">Top</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: right;">
+      <a href="example-creating-udf-iso.html">Example: Creating an ISO with UDF --></a>
+    </div>
+</div>

docs/example-creating-new-basic-iso.md

@@ -0,0 +1,88 @@
+# Example: Creating a new, basic ISO
+
+This example will show how to create a new, basic ISO with no extensions.  Here's the complete code for this example:
+
+```
+try:
+    from cStringIO import StringIO as BytesIO
+except ImportError:
+    from io import BytesIO
+
+import pycdlib
+
+iso = pycdlib.PyCdlib()
+iso.new()
+
+foostr = b'foo\n'
+iso.add_fp(BytesIO(foostr), len(foostr), '/FOO.;1')
+
+iso.add_directory('/DIR1')
+
+iso.write('new.iso')
+iso.close()
+```
+
+Let's take a closer look at the code.
+
+```
+try:
+    from cStringIO import StringIO as BytesIO
+except ImportError:
+    from io import BytesIO
+
+import pycdlib
+```
+
+First import the relevant libraries, including pycdlib itself.
+
+```
+iso = pycdlib.PyCdlib()
+```
+
+Create a new PyCdlib object.  At this point, the object can only do one of two things: open up an existing ISO, or create a new ISO.
+
+```
+iso.new()
+```
+
+Create a new ISO using the [new](pycdlib-api.html#PyCdlib-new) method.  The [new](pycdlib-api.html#PyCdlib-new) method has quite a few available arguments, but by passing no arguments, we ask for a basic interchange level 1 ISO with no extensions.  At this point, we could write out a valid ISO image, but it won't have any files or directories in it, so it wouldn't be very interesting.
+
+```
+foostr = b'foo\n'
+iso.add_fp(BytesIO(foostr), len(foostr), '/FOO.;1')
+```
+
+Now we add a new file to the ISO.  There are a few details to notice in this code.  The first detail is that there are two related APIs called [add_file](pycdlib-api.html#PyCdlib-add_file) and [add_fp](pycdlib-api.html#PyCdlib-add_fp).  The [add_file](pycdlib-api.html#PyCdlib-add_file) API takes the pathname to a file on the local disk to get the contents from.  The [add_fp](pycdlib-api.html#PyCdlib-add_fp) API takes a file-like object to get the contents from; this can be a normal file-object (such as that returned by standard python [open](https://docs.python.org/3.6/library/functions.html#open)), or this can be any other object that acts like a file.  In this case, we use a python [BytesIO](https://docs.python.org/3/library/io.html#binary-i-o) object, which behaves like a file-object but is backed by a string.  The second detail to notice is that the second argument to [add_fp](pycdlib-api.html#PyCdlib-add_fp) is the length of the content to add to the ISO.  Since file-like objects don't have a standard way to get the length, this must be provided by the user.  The [add_file](pycdlib-api.html#PyCdlib-add_file) API can use the length of the file itself for this purpose, so the second argument isn't required there.  The third detail to notice is that the final argument to [add_fp](pycdlib-api.html#PyCdlib-add_fp) is the location of the file on the resulting ISO (also known as the `iso_path`).  The `iso_path` is specified using something similar to a Unix file path.  These paths differ from Unix file paths in that they *must* be absolute paths, since PyCdlib has no concept of a current working directory.  All intermediate directories along the path must exist, otherwise the [add_fp](pycdlib-api.html#PyCdlib-add_fp) call will fail (the `/` root directory always exists and doesn't have to be explicitly created).  Also note that ISO9660-compliant filenames have a slightly odd format owing to their history.  In standard ISO interchange level 1, filenames have a maximum of 8 characters, followed by a required dot, followed by a maximum 3 character extension, followed by a semicolon and a version.  The filename and the extension are both optional, but one or the other must exist.  Only uppercase letters, numbers, and underscore are allowed for either the name or extension.  If any of these rules are violated, PyCdlib will throw an exception.
+
+```
+iso.add_directory('/DIR1')
+```
+
+Here we add a new directory to the ISO called `DIR1`.  Like [add_fp](pycdlib-api.html#PyCdlib-add_fp), the `iso_path` argument to [add_directory](pycdlib-api.html#PyCdlib-add_directory) is an absolute, Unix like pathname.  The rules for ISO directory names are similar to that of filenames, except that directory names do not have extensions and do not have versions.
+
+```
+iso.write('new.iso')
+```
+
+Now we finally get to write out the ISO we just created.  The process of writing out an ISO is sometimes called "mastering".  In any case, this is the process of writing the contents of the ISO out to a file on disk.  Similar to the [add_file](pycdlib-api.html#PyCdlib-add_file) and [add_fp](pycdlib-api.html#PyCdlib-add_fp) methods, there are the related [write](pycdlib-api.html#PyCdlib-write) and [write_fp](pycdlib-api.html#PyCdlib-write_fp) methods, the former of which takes a filename to write to, and the latter of which takes a file-like object.
+
+```
+iso.close()
+```
+
+Close out the PyCdlib object, releasing all resources and invalidating the contents.  After this call, the object can be reused to create a new ISO or open up an existing ISO.
+
+---
+
+<div style="width: 100%; display: table;">
+  <div style="display: table-row;">
+    <div style="width: 33%; display: table-cell; text-align: left;">
+      <a href="examples.html"><-- Examples</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: center;">
+      <a href="https://clalancette.github.io/pycdlib/">Top</a>
+    </div>
+    <div style="width: 33%; display: table-cell; text-align: right;">
+      <a href="example-opening-existing-iso.html">Example: Opening an existing ISO --></a>
+    </div>
+</div>