This article is more of a primer on the history of writing asynchronous code with pure vanilla JavaScript. Feel free to skip to the end if you are solely interested in async functions with ES8 (a.k.a. ES2017). All of the examples below depend on each other. You should be able to follow along by pasting them one after the other into a file. The required new Javascript functionalities are available in Node.js 8 and any recent version of Chrome, Firefox, Safari, Opera and Edge.

Asynchronous programming (that is: off-loading work from a main execution to a sub operation) in JavaScript is archived by passing a function to another function (also known as higher order function). That function is called a callback (or handler) and if the work in the sub operation is complete it will simply call the callback to return it’s results.

This is a big deal in a single threaded language like JavaScript. Without it the main execution would block while the sub operation is active. This would result in poor performance and bad user experience. Examples for asynchronous operations are (depending on the environment): timers, network requests, file requests, event loops…

The test lab:

Lets start with a simple asynchronous function for a fake web request. This function takes two input parameters: an URL and a callback function. If the URL is valid (in this simple example it is valid if it starts with ‘https://’) we would invoke the callback to return the retrieved data after some time, otherwise we will invoke the callback with an error.

/**
 * simulates a fake web request
 * @param {string} the URL, has to start with https://
 * @param {callback} the callback function, this function gets two parameters: err and data
 */
function getData(url, callback) {
  if (!url || !url.startsWith('https://')) {
    callback(`invalid url: ${url}`, undefined);
  } else {
    setTimeout(() => callback(undefined, url), 1000);
  }
}

Now lets invoke this asynchronous function. We invoke the function with an URL and callback. In our case the callback function takes two parameters, an error parameter that indicates if an error occurred and the actual data that our function returned.

getData('https://server/api/data',
  // our callback function with the signature err and data
  function(err, data) {
    // a non undefined err parameter means that an error occurred
    if (err) {
      console.log(`an error occurred: ${err}`)
    // otherwise here is the data from our network request
    } else {
      console.log(`API returned ${data}`);
    }
  }
);

So far so good. But things start getting messy if we expand this to a more real life example. So lets assume that we want to do three web requests in row. In this case we would have to nest the requests in each others success handler. The resulting code structure gets quite difficult to read. This is sometimes called “callback hell”.

To be honest I have hardly seen something like this in a productive system – every sane developer would at least refactor this into separate functions. But even if you do that the program flow is quite difficult to understand. This makes it error-prone and hard to maintain.

// store all required URLs in an array 
const urls = ['https://server/api/data', 'https://server/api/moredata', 'https://server/api/evenmoredata']

// request data from first URL
getData(urls[0], 
  // first callback handler
  function(err, data) {
    if (err) {
      console.log(`an error occurred: ${err}`)
    } else {
      console.log(`API returned ${data}`);
      // request data from second URL
      getData(urls[1],
        // second callback handler
        function(err, data) {
          if (err) {
            console.log(`an error occurred: ${err}`)
          } else {
            console.log(`API returned ${data}`);
            // request data from third URL
            getData(urls[2],
              // third callback handler
              function(err, data) {
                if (err) {
                  console.log(`an error occurred: ${err}`)
                } else {
                  console.log(`API returned ${data}`);
                }
              }
            )
          }
        }
      )
    }
  }
)

To the rescue: Promises

Promises were introduced with ES6 (although the concept is by far older and there are numerous other libraries that implement promises for older JavaScript versions). Promises make handling asynchronous code much easier and cleaner.

To use promises we have to convert our asynchronous function into a promise first. A promise is basically a proxy object for a value that is not known when the proxy is created. This is probably one of the things that is confusing at first. Instead of returning a value we return a promise.

We create a new promise from the prototype by passing a function to it. That function has two predefined callback functions that we can call once our asynchronous function has completed execution:

  • to resolve the promise we call the first callback and pass the result data to it
  • to reject the promise we call the second callback with an error object
/**
 * simulates a fake web request with a promise
 * @param {string} the URL, has to start with https://
 * @return {promise} the promise
 */function getDataPromise(url){
  return new Promise(
    // the function passed to the promise
    function(resolve, reject) {
      if (!url || !url.startsWith('https://')) {
        // reject the promise by calling the reject callback
        reject(`invalid url: ${url}`);
      } else {
        // asynchronously resolve the promise by calling the resolve callback
        setTimeout(() => resolve(url), 1000);
      }
    }
  )
};

To use our new promise we create it and then we bind to its handlers via the methods “then” and “catch”. These methods are chainable.

getDataPromise('https://server/api/data')
// the promise was resolved, process the result
.then(function(data){ 
    console.log(`API returned ${data}`)
})
// the promise was rejected, handle the error
.catch(function(err){
  console.log(`an error occurred: ${err}`);
});

With promises our example with three requests in a row becomes much easier to read. We only need to bind to rejections once since promise rejections propagate through the chain.

getDataPromise(urls[0])
// first promise resolved
.then(function(data){
    console.log(`API returned ${data}`)
    return getDataPromise(urls[1]);
})
// second promise resolved
.then(function(data){ 
    console.log(`API returned ${data}`)
    return getDataPromise(urls[2]);
})
// third promise resolved
.then(function(data){
    console.log(`API returned ${data}`)  
})
// catch all promise rejections
.catch(function(err){
  console.log(`an error occurred: ${err}`);
});

Promises make it much easier to manage the state of multiple asynchronous operations. If we want to run our three request in parallel we could do so as well. For this we would put all our promises into an array and then we create a new promise that resolves once all our promises are complete.

// create an array with the three request promises
let allPromises = [0,1,2].map((index) => getDataPromise(urls[index]));

// create a new promise from the array of promises
Promise.all(allPromises)
// the new promise resolves once all promises in the array are resolved
.then(function(arrayOfResults) {
  // the data of the promise contains the result of all promises as array
  arrayOfResults.forEach((data) => console.log(`API returned ${data}`));
})
// gets called when one of the promises was rejected
.catch(function(err){
  console.log(`an error occurred: ${err}`);
});

Just think of doing this without promises. We would have to implement some kind of countdown latch to detect when all results are available. And then we would have to manage all the error cases. This would result in a lot of additional error prone code that is not part of our business logic.

Promises can help a lot with structuring asynchronous operations but still the underlying mechanism of thinking in handlers remains. If only there was a way to write asynchronous code in a synchronous fashion…

Wait no more: Async functions are here

Finally with ES8, async functions are available and they are one of the best things that happened to JavaScript in years. Async functions leverage some of the concepts of ES6: Promises and Generators. With this we can finally call asynchronous operations as if they were synchronous. For this to work we simply prefix the calling function with the “async” keyword and we place an “await” in front of the promise.

Now our code looks like synchronous code – we don’t need any handlers any more. Results are simply returned as “return value” and exceptions are handled with a simple try catch block.

// to use async functions we have to use the "async" prefix
async function exampleAsyncSingle(){
  try {
    // create the promise with the "await" keyword, execution will await here
    // until the promise is resolved
    let data = await getDataPromise('https://server/api/data');
    console.log(`API returned ${data}`);
  // if the promise rejects, an error is thrown which we can catch
  } catch (err) {
    console.log(`an error occured: ${err}`);    
  }
}
exampleAsyncSingle();

Now lets see how this works out with our three requests in a row:

async function exampleAsyncMultiple(){
  try {
    let data = await getDataPromise(urls[0]);
    console.log(`API returned ${data}`);
    data = await getDataPromise(urls[1]);
    console.log(`API returned ${data}`);
    data = await getDataPromise(urls[2]);
    console.log(`API returned ${data}`);
  } catch (err) {
    console.log(`an error occurred: ${err}`);    
  }
}
exampleAsyncMultiple();

Just how cool is that? So much more readable and easier to maintain. This obviously also works if we have parallel requests:


async function exampleAsyncParallel(){
  try {
    let allPromises = [0,1,2].map((index) => getDataPromise(urls[index]));
    // execution will await here until all promises are resolved
    let arrayOfResults = await Promise.all(allPromises);
    arrayOfResults.forEach((data) => console.log(`API returned ${data}`));
  } catch (err) {
    console.log(`an error occurred: ${err}`);    
  }
}
exampleAsyncParallel();

Conclusion:

Handling asynchronous flow of operations that depend on each other has always been a challenge with JavaScript. Yes, there are other ways to deal with this problem (such as Observables and Fibers). But with ES8 Async Await we finally have a clean way to do this with plain JavaScript.

Keep in mind though that asynchronous operations are usually asynchronous for a reason: to prevent the main execution context from blocking. Async Await lets us write consecutive asynchronous operations in a very readable way. But we still have to think carefully about the Promises underneath to prevent loss of performance or even blocking: which asynchronous operations depend on each other, which operations should run in parallel and which operations are completely independent. With Promises and Async Await we have a pretty good tool belt to deal with all these situations.

From Callbacks and Promises to Async Functions with ES8
Tagged on:                         

4 thoughts on “From Callbacks and Promises to Async Functions with ES8

  • July 26, 2017 at 16:35
    Permalink

    I have a different view on async/await. It’s not that it makes you write asynchronous code in a synchronous fashion. You actually write perfectly synchronous code with all the conveniences and inconveniences that come with it. Only the top async function itself is asynchronous. The await calls within it are perfectly synchronous.

    I always found that it was magical that it could look so much synchronous while being asynchronous. I realized that it was actually not asynchronous at all – no magic, after I read this https://medium.com/@bluepnume/even-with-async-await-you-probably-still-need-promises-9b259854c161 . It explains that within an async function, using await, we get all the inconveniences of synchronous code. Strangely, the author still does not say that the calls that use await to get a value are synchronous calls.

    The confusion comes perhaps from the fact that a computation that uses asynchronous calls to get a value is itself necessarily asynchronous. This is forced. However, the converse is not true: the async function can be asynchronous while making perfectly synchronous calls. It may be that it’s a question of view point on what is an asynchronous call, but I cannot see any way in which it makes sense to say that these await calls are asynchronous.

    The explanation is perhaps that these await calls are based on intermediary promises, which are asynchronous objects. But that would be a weak argument, because the intermediary promise is only the internal implementation – the overall call can itself be synchronous.

    The async function is asynchronous because it is based on a generator that is called with the next method, which returns immediately, before any subsequent next method is executed. However, in the remainder of the computation, which have been started, the await calls are synchronous: these calls have to end, return their value, before we resume the computation.

    To bring out this fact, I have proposed an implementation of async/await without promises : https://medium.com/@dominic.mayers/async-await-without-promises-725e15e1b639 . A promise has three states: pending, resolved, rejected. Because the await calls are synchronous, the pending state is not useful. Therefore, the proposed implementation uses try objects, which only have the two last states and are synchronous objects.

    • July 26, 2017 at 20:05
      Permalink

      You are right, the await calls are perfectly synchronous. Thanks for making that more clear. Because of this we can treat asynchronous operations with “await” inside of the async function as if they were synchronous.

      For a lot of scenarios that makes the code far more readable (i.e. a Node.js Backend that answers a request by issuing a few asynchronous request to some other resources).

      • July 26, 2017 at 20:50
        Permalink

        Yes sure, often only the top level asynchronous call needs to be asynchronous. So, it’s very nice to be able to easily call otherwise asynchronous functions in a perfectly synchronous way. However, some times, the internal calls also need to be asynchronous. Then, to avoid losing performance, one must be careful not to use await in those cases.

        • July 29, 2017 at 19:26
          Permalink

          True. I’ll add a note on this in the conclusion.

Comments are closed.