For developers using Steem APIs, most resort to libraries that can interface between the language of their choice for their application (e.g. Javascript, Python) and the steem blockchain. Trying to do things yourself can lead to weird results, or worse, a poor implementation that does work -- but might break later without knowing why.
As an example of this weird, inconsistent API behaviour, consider the following JSON-RPC calls.
'{"id":"0","jsonrpc":"2.0","method":"call","params":["database_api","get_dynamic_global_properties",[]]}'
'{"id":"0","jsonrpc":"2.0","method":"database_api.get_dynamic_global_properties", "params": {}}'
'{"id":"0","jsonrpc":"2.0","method":"call","params":["database_api","get_dynamic_global_properties",{}]}'
'{"id":"0","jsonrpc":"2.0","method":"database_api.get_dynamic_global_properties"}'
'{"id":"0","jsonrpc":"2.0","method":"call","params":[0,"get_dynamic_global_properties",[]]}'
'{"id":"0","jsonrpc":"2.0","method":"call","params":[0,"get_dynamic_global_properties",{}]}'
All of the above are valid calls to Jussi -- the Steemit provided API middleware -- and all return the same result. I’m even being generous here, as there are a lot more valid calls; for example, the value for “id” can be anything, the json keys can be provided in any order, and there is also a condenser_api call that can also be used in many ways to obtain the same result -- with all of the this, there is a near limitless number of valid calls to achieve the exact same simple data.
Here's a fun question, why is this call not valid but the above are?
'{"id":"0","jsonrpc":"2.0","method":"database_api.get_dynamic_global_properties", "params": []}'
And, out of all the above example calls, only the following are actually valid direct calls to a steemd API node:
'{"id":"0","jsonrpc":"2.0","method":"database_api.get_dynamic_global_properties", "params": {}}'
'{"id":"0","jsonrpc":"2.0","method":"call","params":["database_api","get_dynamic_global_properties",{}]}'
'{"id":"0","jsonrpc":"2.0","method":"database_api.get_dynamic_global_properties"}'
Yes, the Steem API is a clusterfuck.
JSON-RPC vs REST
Even worse, the above calls are a JSON-RPC "POST" request. What this means is that the operator is sending (“POST”ing) information to the API: saying what method to execute, and what parameters to execute with. For many types of APIs this format can make a lot of sense; methods or parameters may be complex. However, for public facing APIs, the industry standard is not JSON-RPC, but rather what is known as “RESTful” requests. For these types of requests, a differentiation is made that the requested method and parameters are encoded in the URL itself.
As an example, take the following:
https://api.coinmarketcap.com/v1/ticker/bitcoin/
This API from CoinMarketCap gives you a result based on the ticker requested: in this case, bitcoin. When you compare this type of API to the one we currently have for steem, it’s pretty clear which one is more developer friendly.
Designing a RESTful API for Steem
As many people know, I’ve been playing around with steem RPCs and API nodes for a while now. I run a full API node at https://anyx.io , and serve as the main API provider many of the leading steem apps like Busy, Partiko, and Steemmonsters. Notably, this infrastructure is owned, hosted, and operated by me. You can read more about that here and here.
I’ve recently been working on a layer that will accept RESTful requests and convert them into JSON-RPC requests that go to a steemd node. The long-term idea will be able to replace the API in steemd itself, but this will be a “beta” version interpreter to speculate on how the new format would look like.
The layer I have created basically takes the traditional appbase APIs and listens like it was a REST implementation. Some things that aren't supported are the (sort-of deprecated) condenser_api, nested JSON requests, and actual POST data (like pushing signed transactions). I don’t plan to support condenser_api in this format, as honestly it needs to go. (TL;DR: it doesn’t differentiate between actual steemd plugins and gives different results based on which plugins are on the steemd node. Appbase differentiates requests based on plugins.)
How do requests work in this format?
In general, to convert from JSON-RPC format, we build a URL as follows: take the appbase plugin first, followed by the appbase method, then provide any required parameters in URL query format. That’s really all there is to it!
Non-parameter examples:
https://anyx.io/v1/database_api/get_active_witnesses
https://anyx.io/v1/database_api/get_config
https://anyx.io/v1/database_api/get_dynamic_global_properties
Parameters specified (of which you can specify multiple times to make an array request):
https://anyx.io/v1/database_api/find_accounts?accounts=anyx
https://anyx.io/v1/database_api/find_witnesses?owners=anyx&owners=jesta&owners=timcliff
Some fun examples of history:
https://anyx.io/v1/block_api/get_block?block_num=10000000
https://anyx.io/v1/account_history_api/get_ops_in_block?block_num=10000000&only_virtual=false
https://anyx.io/v1/account_history_api/get_account_history?account=anyx&start=-1&limit=10
Some more useful examples:
https://anyx.io/v1/follow_api/get_account_reputations?account_lower_bound=anyx&limit=1
https://anyx.io/v1/follow_api/get_blog?account=anyx&start_entry_id=0&limit=1
https://anyx.io/v1/follow_api/get_follow_count?account=anyx
https://anyx.io/v1/rc_api/find_rc_accounts?accounts=anyx
https://anyx.io/v1/tags_api/get_discussion?author=cheetah&permlink=faq-about-cheetah
And finally, lets bring things full circle. As an example of how much more sane it is for a developer, these two requests are equivalent:
As specified by the docs:
curl -s --data '{"jsonrpc":"2.0", "method":"database_api.get_dynamic_global_properties", "id":1}' https://api.steemit.com
And remember how many options there were for that data payload?
Using our REST layer, there’s only one simple way to get that data:
curl https://anyx.io/v1/database_api/get_dynamic_global_properties
Or hell, just click the link and your browser can display it.
Designing new Middleware?
Steemit has spent a lot of time designing around Steems’ API format. The reason why a custom solution was needed is that there's a need to both parse and understand the JSON RPC request first, in order to understand how to serve it. You couldn't simply scale out servers because you would need to read and interpret the request data before routing it to a server that can handle it.
Swapping over to the RESTful implementation would be a big change: there would no longer be a need for Jussi with this format, as an http server like NGINX can now correctly route based on location directives. Furthermore, traditional systems for caching RESTful requests can be used. No more custom solutions would be needed -- we can move completely over to using well-established open source solutions like NGINX and REDIS directly without writing any custom middleware.
Steps to move forward
Notably, this is not a 'release' announcement of this RESTful layer. The implementation is still very much in the design phase as a work in progress, and there are likely bugs or inconsistencies, and the format is subject to change. This post is instead targeted at curious developers who want to see what other solutions to Steems’ API could look like. In general, I’m looking at getting feedback from these devs.
Mostly, I'd like to know if:
- Do you like this format and design -- is it worth supporting/continuing developing?
- If you're an app dev, would you swap over to using this format compared to using the existing libraries?
- Any comments or suggestions on the format?
Furthermore, if you’re interested in toying around, there’s a few more things I’d be looking for:
- Can you break/crash things?
- Do certain API calls not work?
- Would you be interested in custom API requests for your application?
As an example of (c), I’ve been testing API requests like “given a timestamp, give me the block number” or “what was the original content / diff of this post”. These kinds of API calls can be done without indexing or changing steemd, but can rather be implemented in the API layer to perform some server-side logic to reduce application-side logic.
Personally, I love having logical URLs for requests, rather than having to use curl with POST requests to quickly get data I'm interested in. If I want to get the contents of a block, I can now do so without opening the command line, and rather simply use a browser. It makes so much more sense to be able to link to blockchain data that will be obtained directly from an API node.
I’m hoping that there will be enough interest in this format to push API’s in this direction.
Like what I'm doing for Steem? You can read more about my witness candidacy here:
https://steemit.com/witness/@anyx/updated-witness-application
Then please consider voting for me as a witness here!
https://steemit.com/~witnesses