OysterStorage Process

Hey'all!

Today we will be having a further look into the Oyster Protocol product "Oyster Storage".

Oyster Storage lets the end user upload & download files on the IOTA tangle. The uploaded file is split into parts and encrypted locally in the browser. This isolation ensures the impossibility of a malicious actor retrieving the data since it can only be accessed with the corresponding encryption key, which is known as the Oyster Handle.

"The development team has been doing some pre-deployment QA before Oyster rolls out the badger DB changes which will reduce bottlenecks on the backend and help increase the maximum file size we can support (note that the upcoming deployment of changes will not include the front-end changes which are also needed to support larger file sizes). The development team was able to upload a 200mb file using Oyster’s backend alone. Further QA is needed on the client-side of the Oyster Storage web interface before the current 5mb file size limit will be increased." (Source: https://medium.com/oysterprotocol/development-update-july-13th-2018-ac975ca973b6)

UPLOAD PROCESS
To upload a file via the Oyster Protocol, the Storage User's client selects two Broker Nodes to commit the data to the Tangle. The data is processed from the beginning and end of the file, one Broker Node performing each, similar to a candle that burns on both ends.

Once the file has been fully committed to the Tangle, the client begins downloading the entire Data Map in order to verify it's integrity.

Each submitted transaction must perform Proof of Work for two prior transactions, therefore confirming them. These two transactions are contextually referenced as the branch and trunk. Each transaction has a payload capacity which is used to retain the data that is uploaded by the Storage User. Transactions arepropagated throughout a mesh-net of Nodes that have mutually peered with each other, whilst each Node retains a redundant copy of the transactions. This leads to a great redundancy of data copies, therefore heavily mitigating the risk of data-loss whilst not relying on a centralized hosting provider.

Once the transaction is confirmed, the upload process will beginn. The upload takes several minutes for a smaller file, e.g. a photo, respectively longer for a larger file (= 5 MB).

Data is stored on the Tangle in ~1 KB parts within the transaction payload. A SHA256 hash is the referenced basis for storing and retrieving data on the Tangle. When a SHA256 hash has been selected to represent data, it is converted into it's trinary form to represent the recipient address of the transaction.

When to upload is completed, the following message will show: "Your file has been successfully uploaded to the Tangle. An Oyster handle has been generated below. This handle is the only way to access your file on the Tangle. Please store this handle in a safe place."

The Handle Explained

The first 8 characters of the Oyster Handle represents the name of the file. This is usually copied from the filename that was uploaded to the browser, but can also be customized by the Storage User for their own references. The Primordial Hash is a 64 character long SHA256 hash of random input that is generated from within the Storage User's browser with as much entropy as possible. The last 8 characters of the Handle is the cryptographic salt that differentiates the Primordial Hash from the overall encryption key. The salt is used to further protect the data in case the Primordial Hash is found because of a future weakness in a hash function or a rainbow table attack on the Genesis Hash. Therefore the entire 80 character long Handle is the entire encryption key used to encrypt and decrypt the split parts of the data.

DOWNLOAD PROCESS
To retrieve the data from the Tangle the hash is again converted into it's trinary form to produce the recipient address, and then all of the transactions under the address are recovered. The transaction with the oldest issuance timestamp contains the payload data that represents the selected hash.