Asyncis in the Enterprise Collaboration Industry

Async

async

One of my favorite interview questions is “What do such words as async and await tell you?” because it opens up an opportunity to have an interesting discussion with an interviewee… Or it doesn’t because they float on this topic. In my opinion, it is drastically important to understand why we use this technique. I feel like many developers prefer to rely on the “it is the best practice” statement and use asynchronous methods blindly. This article shows the difference between asynchronous and synchronous methods in practice. Tools .NET Web API application (test target) 2 Azure SQL Databases 2 Azure App Service on Windows (hosts the application) Azure App Insights (to gather metrics) locust framework (to simulate user load). Configuration I will run a benchmark in the following way. Two independent locust instances are running on two machines. Locust instances simulate a user that does the following: The user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, and stays idle for 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment. The user from the locust host 2 behaves precisely the same, with only one difference — he hits the asynchronous endpoint of App Service 2. Under the hood, each App Service connects to its own database and executes a SELECT query that takes five seconds and returns a few rows of data. See the controller’s code below for references. I’ll use Dapper to make a call to the database. I’d like to draw your attention to the fact that the asynchronous endpoint calls the database asynchronously as well (QueryAsync<T>). It's worth adding that I deploy the same code to both app services. During the test, the number of users grows evenly to the target number (Number of Users). The speed of growth is controlled by a Spawn Rate parameter (number of unique users to join per second) — the higher the number is, the quicker the users are being added. The spawn rate is set to 10 users/s for all experiments. All experiments are limited to 15 minutes. You may find machine configuration details in the Technical Details section of the article. Metrics requests per minute — shows the number of requests that the application actually processed and returned a status code.
thread count — shows the number of threads the app service consumes.
median response time, ms The red lines refer to asynchronous, and the blue lines — to the synchronous endpoint, respectively. That’s it about theory. Let’s start. Experiment #1 number of users: 75 (per service) We can see that both endpoints perform similarly — handling around 750 requests per minute with a median response time of 5200 ms. The most fascinating graph in this experiment is a thread trend. You can see significantly higher numbers for the synchronous endpoint (a blue graph) — more than 100 threads! That’s expected, though, and matches the theory — when a request comes in, and the application makes a call to the database, the thread is blocked because it has to wait for a roundtrip to complete. Therefore, when another request comes in, the application has to produce a new thread to handle it. The red graph — the asynchronous endpoint thread count— proves different behavior. When a request comes in and the application makes a call to the database, the thread returns to a thread pool instead of being blocked. Therefore, when another request comes in, this free thread is reused. Despite incoming requests growing, the application doesn’t require any new threads, so their count remains the same. It’s worth mentioning the 3rd metric — median response time. Both endpoints showed the same result — 5200 ms. So, there is no difference in terms of performance. Now, it is time to pull up the stakes. Experiment #2 number of users: 150 We doubled the load. The asynchronous endpoint handles this task successfully—its request per minute rate floats around 1500. The synchronous brother eventually reached a comparable number of 1410. But if you look at the graph below, you will see that it took 10 minutes! The reason is that the synchronous endpoint reacts to a new user's arrival by creating another thread, but users are being added to the system (just to remind you that the Spawn Rate is 10 users/s) quicker than the web server can adapt. That’s why it queued so many requests at the very beginning. Unsurprisingly, the thread count metric is still around 34 for the asynchronous endpoint, while it increased from 102 to 155 for the synchronous one. The median response time degraded similarly to the request per minute rate — the synchronous response time was much higher at the beginning of the experiment. If I had kept the test for 24 hours, the median numbers would become even. Experiment #3 number of users: 200 The third experiment is intended to prove trends revealed during the second one —we can see further degradation of the synchronous endpoint. Conclusion Using asynchronous instead of synchronous operations doesn’t directly improve performance or user experience. First, it enhances stability and predictability under pressure. In other words, it raises the load threshold so the system can process more before it degrades. Appendix #1. Technical Details Azure App Service: B1, 100 ACU, 1,75 Gb Memory, A-Series compute equivalent.
Azure SQL Database: Standard S4: 200 DTUs, 500 Mb storage.
SQL Connection settings: Max Pool Size=200. Appendix #2. Notes To achieve the cleanest test result, I should have run tests from 2 VMs located in the same network where the target App Services sit. However, I assumed that a network lag would impact both apps in a more or less similar way. Therefore, it cannot jeopardize the main goal — comparing how asynchronous and synchronous methods behave. Appendix #3. Bonus Experiment What did I hack to force the synchronous endpoint to perform almost as asynchronous and plot the graph below (experiment conditions are the same as in the third one — 200 users)? One of my favorite interview questions is “What do such words as async and await tell you?” because it opens up an opportunity to have an interesting discussion with an interviewee… Or it doesn’t because they float on this topic. In my opinion, it is drastically important to understand why we use this technique. async await I feel like many developers prefer to rely on the “it is the best practice” statement and use asynchronous methods blindly. This article shows the difference between asynchronous and synchronous methods in practice. Tools .NET Web API application (test target) .NET Web API application (test target) 2 Azure SQL Databases 2 Azure SQL Databases 2 Azure App Service on Windows (hosts the application) 2 Azure App Service on Windows (hosts the application) Azure App Insights (to gather metrics) Azure App Insights (to gather metrics) locust framework (to simulate user load). locust framework (to simulate user load). locust Configuration I will run a benchmark in the following way. Two independent locust instances are running on two machines. Locust instances simulate a user that does the following: The user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, and stays idle for 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment. The user from the locust host 1 hits the synchronous endpoint of the App Service 1, gets the response, and stays idle for 0.5–1 seconds (the exact time delay is random). Repeats till the end of the experiment. synchronous App Service 1, The user from the locust host 2 behaves precisely the same, with only one difference — he hits the asynchronous endpoint of App Service 2. The user from the locust host 2 behaves precisely the same, with only one difference — he hits the asynchronous endpoint of App Service 2. asynchronous App Service 2. Under the hood, each App Service connects to its own database and executes a SELECT query that takes five seconds and returns a few rows of data. See the controller’s code below for references. I’ll use Dapper to make a call to the database. I’d like to draw your attention to the fact that the asynchronous endpoint calls the database asynchronously as well ( QueryAsync<T> ). QueryAsync<T> It's worth adding that I deploy the same code to both app services. During the test, the number of users grows evenly to the target number ( Number of Users ). The speed of growth is controlled by a Spawn Rate parameter (number of unique users to join per second) — the higher the number is, the quicker the users are being added. The spawn rate is set to 10 users/s for all experiments. Number of Users Spawn Rate spawn rate All experiments are limited to 15 minutes. You may find machine configuration details in the Technical Details section of the article. Metrics requests per minute — shows the number of requests that the application actually processed and returned a status code. thread count — shows the number of threads the app service consumes. median response time, ms requests per minute — shows the number of requests that the application actually processed and returned a status code. requests per minute thread count — shows the number of threads the app service consumes. thread count median response time, ms median response time, ms The red lines refer to asynchronous, and the blue lines — to the synchronous endpoint, respectively. That’s it about theory. Let’s start. Experiment #1 Experiment #1 number of users: 75 (per service) number of users : 75 (per service) number of users We can see that both endpoints perform similarly — handling around 750 requests per minute with a median response time of 5200 ms. The most fascinating graph in this experiment is a thread trend. You can see significantly higher numbers for the synchronous endpoint (a blue graph) — more than 100 threads! That’s expected, though, and matches the theory — when a request comes in, and the application makes a call to the database, the thread is blocked because it has to wait for a roundtrip to complete. Therefore, when another request comes in, the application has to produce a new thread to handle it. The red graph — the asynchronous endpoint thread count— proves different behavior. When a request comes in and the application makes a call to the database, the thread returns to a thread pool instead of being blocked. Therefore, when another request comes in, this free thread is reused. Despite incoming requests growing, the application doesn’t require any new threads, so their count remains the same. It’s worth mentioning the 3rd metric — median response time . Both endpoints showed the same result — 5200 ms. So, there is no difference in terms of performance. median response time Now, it is time to pull up the stakes. Experiment #2 number of users: 150 number of users : 150 number of users We doubled the load. The asynchronous endpoint handles this task successfully—its request per minute rate floats around 1500. The synchronous brother eventually reached a comparable number of 1410. But if you look at the graph below, you will see that it took 10 minutes! request per minute The reason is that the synchronous endpoint reacts to a new user's arrival by creating another thread, but users are being added to the system (just to remind you that the Spawn Rate is 10 users/s) quicker than the web server can adapt. That’s why it queued so many requests at the very beginning. Spawn Rate Unsurprisingly, the thread count metric is still around 34 for the asynchronous endpoint, while it increased from 102 to 155 for the synchronous one. The median response time degraded similarly to the request per minute rate — the synchronous response time was much higher at the beginning of the experiment. If I had kept the test for 24 hours, the median numbers would become even. thread count median response time request per minute Experiment #3 number of users: 200 number of users : 200 number of users The third experiment is intended to prove trends revealed during the second one —we can see further degradation of the synchronous endpoint. Conclusion Using asynchronous instead of synchronous operations doesn’t directly improve performance or user experience. First, it enhances stability and predictability under pressure. In other words, it raises the load threshold so the system can process more before it degrades. Appendix #1. Technical Details Azure App Service: B1, 100 ACU, 1,75 Gb Memory, A-Series compute equivalent. Azure SQL Database: Standard S4: 200 DTUs, 500 Mb storage. SQL Connection settings: Max Pool Size=200. Azure App Service: B1, 100 ACU , 1,75 Gb Memory, A-Series compute equivalent. ACU Azure SQL Database: Standard S4: 200 DTUs, 500 Mb storage. SQL Connection settings: Max Pool Size=200. Appendix #2. Notes To achieve the cleanest test result, I should have run tests from 2 VMs located in the same network where the target App Services sit. However, I assumed that a network lag would impact both apps in a more or less similar way. Therefore, it cannot jeopardize the main goal — comparing how asynchronous and synchronous methods behave. Appendix #3. Bonus Experiment Appendix #3. Bonus Experiment What did I hack to force the synchronous endpoint to perform almost as asynchronous and plot the graph below (experiment conditions are the same as in the third one — 200 users)?

Async vs Sync Benchmark (.NET): The Difference Between Asynchronous and Synchronous Methods

مقارنة بين Async وSync (.NET): الفرق بين الطرق المتزامنة وغير المتزامنة

Async vs Sync Benchmark (.NET): Asinxron və Sinxron Metodlar Arasındakı Fərq

Async vs Sync Benchmark (.NET): la diferencia entre métodos asincrónicos y sincrónicos

Async vs Sync Benchmark (.NET): a diferenza entre os métodos asíncronos e síncronos

Async vs Sync Benchmark (.NET): Az aszinkron és a szinkron módszerek közötti különbség

Async vs Sync Benchmark (.NET): la differenza tra metodi asincroni e sincroni

非同期 vs 同期ベンチマーク (.NET): 非同期メソッドと同期メソッドの違い

Async vs Sync Benchmark (.NET)៖ ភាពខុសគ្នារវាងវិធីសាស្ត្រ Asynchronous និង Synchronous Methods

Async vs Sync Benchmark (.NET): Асинхрондуу жана синхрондуу методдордун ортосундагы айырма

Async vs Sync Benchmark (.NET): różnica między metodami asynchronicznymi i synchronicznymi

Async vs Sync Benchmark (.NET): Rozdiel medzi asynchrónnymi a synchrónnymi metódami

Async vs Sync Benchmark (.NET): Musiyano uripakati peAsynchronous uye Synchronous Methods

Designed for developers at the intersection of curiosity and professional growth, this primer aims to illuminate how asynchronous programming can elevate application performance, scalability, and responsiveness. By focusing on the universal principles of non-blocking operations and event loops, we venture beyond specific technologies like AsyncIO, Node.js, or Go. This exploration is crafted for software developers eager to grasp the efficiencies driving modern software without the complexity of technical jargon. Think of how a single thread handling requests is like a single-lane road in a busy city: each request is a car, and when there's only one lane, cars line up, causing delays. But what if our city — our service — could manage traffic more cleverly? That's where the magic of an asynchronous runtime comes into play. It's like adding an intelligent traffic system to our city, guiding cars through multiple lanes and intersections without waiting. This system keeps traffic flowing smoothly, ensuring no car (or task) waits too long, which is precisely how an asynchronous runtime keeps our software running efficiently. Let's consider an example of a service that completes I/O operations synchronously. For clarity, these operations are shown outside the main flow of execution in the diagram: Blocking I/O during your service's startup phase might be acceptable, but avoiding this approach when processing external requests is advisable. The following diagram illustrates how service efficiency can be enhanced through the adoption of non-blocking I/O operations: These examples highlight scenarios where the server's performance gains are maximized. Nevertheless, the advantage of non-blocking operations persists across any frequency of incoming requests. Even in less-than-ideal conditions, performance benefits from integrating dedicated I/O threads into the processing workflow. The total time required to process a request (from the client's initial request to the final response, as depicted by the blue number on the right) will invariably decrease, provided there are sufficient threads to handle all requests. In the worst-case scenario, this duration will not surpass that of synchronous processing methods. We then encounter a practice that might seem counterintuitive to many developers. When I/O operations constitute a significant portion of request processing time, optimizing other code segments may yield slight improvement. The duration to fetch data from a cache might align closely with the time dedicated to business logic and template rendering. Employing caching or an in-process database can reduce data retrieval times compared to other processing activities. To segment code effectively and facilitate callback execution, one can instruct the runtime to proceed to the next event loop cycle. Below is an illustrative example of how this concept is applied: // blocking callbacks
function func1_cb(str, cb) {
 var res = func1(str);
 cb(res);
}

function func2_cb(str, cb) {
 var res = func2(str);
 cb(res);
}

// non-blocking callbacks
function func1_cb(str, cb) {
 var res = func1(str);
 process.nextTick(function () {
   cb(res);
 });
}

function func2_cb(str, cb) {
 var res = func2(str);
 process.nextTick(function () {
   cb(res);
 });
}

// usage example
func1_cb(content, function (str) {
 func2_cb(str, function (result) {
   // work with result
 });
}); By adopting this methodology to split two parts of calculations, the overall processing time for scenarios with nearly concurrent request arrivals remains the same. However, the response to the first request is delayed: This scenario represents the least favorable outcome when employing the “next tick” strategy. As shown in the initial example, the “next tick” method proves benign for infrequent requests. Should requests arrive at a moderate pace, leveraging this technique enhances processing speed by allowing the initiation of new requests and the start of non-blocking operations to be interleaved during execution breaks. This approach effectively reduces both the total processing time and the average time per request: In conclusion, adopting non-blocking I/O is crucial for enhancing application performance and beneficial in environments with both sparse and heavy incoming request volumes. Additionally, effectively sequencing the execution flow — illustrated by concepts similar to the "next tick" technique — significantly improves server efficiency. Embracing these asynchronous programming practices offers clear advantages over traditional, synchronous methods.

Asynchronous Runtimes: A Primer for the Perplexed Developer

Asynchronous Runtimes: A Primer for the Perplexed ডেভেলপার

Asynchrone Laufzeiten: Eine Einführung für den verwirrten Entwickler

Runtimes asíncronos: un primer para los desarrolladores perplejos

Environnements d'exécution asynchrones : un guide pour les développeurs perplexes

Asynchronous Runtimes: एक भ्रमित डेवलपर्स के लिए एक प्राथमिक

非同期ランタイム: 困惑する開発者のための入門書

Asynchronous Runtimes: A Primer for the Perplexed 개발자

Tempos de execução assíncronos: um guia para o desenvolvedor perplexo

Асинхронные среды выполнения: практическое руководство для озадаченных разработчиков

Asynchronous Runtimes: A Primer for the Perplexed Geliştirici

Thời gian chạy không đồng bộ: Một bài học vỡ lòng cho nhà phát triển bối rối

异步运行时：困惑的开发人员的入门指南

Async await in is a feature that allows you to write asynchronous code that is easy to read and maintain. It greatly simplifies how you handle asynchronous operations by providing a structured way to await tasks and their results. With the rate at which computers are scaling performance & cores, and our growing need to interface with many external services, async await in C# can make our developer lives easier. C# In the following sections, I’ll highlight 3 beginner tips for using async await in C#. Each tip will be accompanied by a code example, allowing you to see these concepts in action. These cancellation, and performance optimization, so you’ll be off to the races writing async code. tips will cover important aspects such as error handling, Tip 1: Understand the Basics of async await in C# Async await in C# are two special keywords that are used to handle asynchronous code execution. Asynchronous programming enables us to write C# code that can perform multiple operations concurrently, improving the overall performance and responsiveness of an application. In C#, a task represents an asynchronous operation and it’s used to encapsulate a unit of work that can be executed concurrently with other tasks. When using async/await, you can use tasks to handle and manage asynchronous operations in a structured way. To define an async method in C#, you need to add the async modifier to the method signature. For example: public async Task DoAsyncWork()
{
    // Asynchronous code here
} In the above example, the method is declared as async and returns a . This indicates that the method will perform asynchronous operations and can be awaited. DoAsyncWork Task Within the async method, you can use the await keyword to signal the point where the execution should wait for the completion of an asynchronous operation. For example: public async Task DoAsyncWork()
{
    // Perform asynchronous operations
    await Task.Delay(1000);

    // Continue with other operations
} In the above example, the statement pauses the execution of the method until the operation completes. This allows other code, such as other async tasks, to execute concurrently. By using tasks, you can of asynchronous programming in C#. await Task.Delay(1000) Task.Delay harness the power Tip 2: Handling Multiple Exceptions with async await . When working with async code in C#, handling exceptions can present some challenges due to the asynchronous nature of the code. In certain situations, multiple exceptions can occur when executing async code. It’s important to be able to handle and aggregate these exceptions effectively. To handle multiple exceptions, C# provides the class, which allows us to group and manage multiple exceptions as a single entity. Exception handling is an important aspect of writing any software application AggregateException When multiple exceptions occur in an async operation, the class collects these exceptions and allows us to access and handle them in a centralized manner. This can be particularly useful for scenarios where we need to process all exceptions generated during an async operation. To handle exceptions using the class, we can wrap the code that may throw exceptions in a try-catch block. Within the catch block, we can access the property of the to access each individual exception that occurred. AggregateException AggregateException InnerExceptions AggregateException Here’s an example of how to use the class to handle and aggregate exceptions in async code: AggregateException try
{
    await SomeAsyncOperation();
}
catch (AggregateException ex)
{
    foreach (var innerException in ex.InnerExceptions)
    {
        // Handle each individual exception
        Console.WriteLine($"Exception: {innerException.Message");
    }
} In the example above, represents an async operation that may throw one or more exceptions. By wrapping the statement in a try-catch block and catching an , we are able to access and handle each individual exception that occurred within the async operation. By properly handling and aggregating exceptions in async code using the class, we can ensure that our code can gracefully handle any unexpected errors that may occur during async operations. SomeAsyncOperation await AggregateException AggregateException Tip 3: Avoiding Deadlocks with async await Deadlocks can occur when two or more tasks are waiting for each other to complete indefinitely, resulting in a state where none of the tasks can proceed. Stop Blocking Your Async Calls! One of the most common causes of deadlocks in async methods includes using the or methods inside an async method, which blocks the main thread and can lead to deadlocks. The best advice for avoiding deadlocks with async code: Task.Result Task.Wait “Don’t block on async code” Stephen Cleary He points out in his article that this tip (stop blocking) or the tip that immediately follows this (configuring the awaiter) are both able to help prevent deadlocks… but relying ONLY on the next pattern is more of a risk. That is, of these two suggestions, making your C# code async await all of the way through is the safer bet. That means instead of having something like: public class MyController(
  IWeatherService _weatherService) :
  ApiController
{
  public string GetWeatherData()
  {
    var weatherData = _weatherService.GetWeatherData(...).Result;
    return Results.OK(weatherData);
  }
} Instead, we’d make the top-level async as well: public class MyController(
  IWeatherService _weatherService) :
  ApiController
{
  public async Task<string> GetWeatherData()
  {
    var weatherData = await _weatherService.GetWeatherData(...);
    return Results.OK(weatherData);
  }
} As soon as we introduce a or call, we are effectively blocking — and that is one of the key situations that can lead to deadlocks as the application gets more complex. .Result .Wait() Using ConfigureAwait(false) with Async Await in C# To prevent deadlocks, the method can be used with the parameter. This instructs the operator to not capture the current synchronization context, allowing the continuation to be executed on any available thread from the thread pool — not just the thread that started the call! ConfigureAwait false await With , we can explicitly state that we do not require the original synchronization context to be captured, preventing the deadlock issue. Keep in mind that using should be done carefully and only when it is absolutely necessary, as it may introduce potential issues if the code relies on the captured synchronization context for specific requirements. For example, if you’re running UI logic and you come back to execution not on the UI thread, you may be in for some surprises! ConfigureAwait(false) ConfigureAwait(false) Let’s take a look at an example to illustrate the usage of to prevent deadlocks: ConfigureAwait(false) public async Task<string> GetDataAsync()
{
    // NOTE: assume we enter this method on some given thread

    // use a false parameter on ConfigureAwait
    await SomeAsyncMethod().ConfigureAwait(false);

    // NOTE: we could be on a different thread than when we started this method!

    return ProcessData();
} In the above example, is awaited using . This allows the continuation of the code to execute on any available thread, ensuring that the synchronization context is not captured and preventing potential deadlocks. SomeAsyncMethod ConfigureAwait(false) And while Stephen Cleary does say that this will help prevent deadlocks, he also had this to add: Using to avoid deadlocks is a dangerous practice. You would have to use for in the transitive closure of all methods called by the blocking code, . Using to avoid deadlock is at best ). ConfigureAwait(false) ConfigureAwait(false) every await including all third- and second-party code ConfigureAwait(false) just a hack Stephen Cleary Wrapping up Async Await in C# In this article, I’ve discussed 3 beginner tips for using async await in C# to write concurrent code. Async await in C# can be a lot to absorb right in the beginning, so it will take some time and practice for it to feel more natural! We got to see the basics of the keyword usage for async await — With this, you can now write asynchronous code! From there, we looked at how multiple exceptions can arise in async await code paths and how to navigate that. And we concluded with some ideas around preventing deadlocks by looking at 2 possibilities we can leverage. Async await in C# is something you’re going to want to learn and practice! If you found this useful and you’re looking for more learning opportunities, consider and check out my ! subscribing to my free weekly software engineering newsletter free videos on YouTube Also published here.

3 Beginner Tips You Need to Know for async await in C#

Handling asynchronous operations in Javascript is a fundamental part of creating responsive and interactive web applications. allows tasks like API calls, file reading, or timers to run in the background, ensuring the application remains responsive. Enter JavaScript Promises, a powerful abstraction for managing these asynchronous operations. Asynchronous programming The Basics of Promises A in JavaScript represents the eventual completion (or failure) of an asynchronous operation and its resulting value. A promise can be in one of three states: Promise : The initial state where the operation has not been completed yet. Pending : The operation was completed successfully. Fulfilled : The operation failed. Rejected Structure of a Promise The basic syntax for creating a promise is: const myPromise = new Promise((resolve, reject) => {
  // Asynchronous operation here
  if (/* operation successful */) {
    resolve('Success');
  } else {
    reject('Error');
  }
}); Creating and Using Promises Creating a promise involves passing an executor function to the Promise constructor, which contains the asynchronous operation and dictates whether the promise should be resolved or rejected. Using Promises Once a promise is created, it can be used with , , and methods to handle the fulfilled or rejected state. then() .catch() .finally() myPromise
  .then((value) => {
    // Handle success
    console.log(value);
  })
  .catch((error) => {
    // Handle error
    console.error(error);
  })
  .finally(() => {
    // Execute cleanup or final operations
    console.log('Completed');
  }); allows you to chain multiple promises, creating a sequence of asynchronous operations that execute one after the other. .then() Error Handling in Promises Proper error handling is crucial in asynchronous programming to ensure the reliability and robustness of web applications. Promises provide a clean and straightforward way to catch and handle errors. Using .catch() The .catch() method is used to handle any errors that occur in the promise chain. fetch('https://api.example.com/data')
  .then(response => response.json())
  .catch(error => console.error('Failed to fetch data:', error)); Error propagation in promises ensures that if an error is not caught by a in the chain, it will bubble up to the next . .catch() .catch() Advanced Promise Features Promises in JavaScript offer more than just basic functionality. There are several advanced features designed to handle complex asynchronous patterns efficiently. Promise.all() When you need to run multiple promises in parallel and wait for all of them to complete, is incredibly useful. Promise.all() Promise.all([promise1, promise2, promise3])
  .then((results) => {
    // results is an array of each promise's result
    console.log(results);
  })
  .catch((error) => {
    // If any promise is rejected, catch the error
    console.error("A promise failed to resolve", error);
  }); Promise.race() is similar to , but it resolves or rejects as soon as one of the promises in the iterable resolves or rejects, with the value or reason from that promise. Promise.race() Promise.all() Promise.race([promise1, promise2, promise3])
  .then((value) => {
    // Value of the first resolved promise
    console.log(value);
  })
  .catch((error) => {
    // Error of the first rejected promise
    console.error(error);
  }); Promise.allSettled() This method returns a promise that resolves after all of the given promises have either been resolved or rejected, with an array of objects that each describe the outcome of each promise. Promise.allSettled([promise1, promise2, promise3])
  .then((results) => {
    results.forEach((result) => console.log(result.status));
  }); Promise.any() takes an iterable of Promise objects and, as soon as one of the promises in the iterable fulfills, returns a single promise that resolves with the value from that promise. Promise.any() Promise.any([promise1, promise2, promise3])
  .then((value) => {
    console.log(value);
  })
  .catch((error) => {
    console.error('All promises were rejected');
  }); Real-world Use Cases for Promises Promises are not just theoretical constructs but have practical applications in everyday programming. Here are a couple of examples: Fetching Data from an API One common use case is fetching data from a remote API: fetch('https://api.example.com/data')
  .then(response => response.json())
  .then(data => console.log(data))
  .catch(error => console.error('Fetch error:', error)); Wrapping Callback-based APIs Another practical use is to wrap callback-based functions (like those in Node.js) into promises for a cleaner, more modern API usage: const fs = require('fs').promises;

fs.readFile('path/to/file.txt', 'utf8')
  .then(data => console.log(data))
  .catch(error => console.error('Read file error:', error)); Promises vs. Async/Await While promises are powerful on their own, the async/await syntax introduced in ES2017 provides a more straightforward way to work with asynchronous operations, making your code cleaner and easier to understand. async function fetchData() {
  try {
    const response = await fetch('https://api.example.com/data');
    const data = await response.json();
    console.log(data);
  } catch (error) {
    console.error('Fetch error:', error);
  }
} async/await is syntactic sugar over promises and can be used wherever promises are used. Best Practices and Common Mistakes To make the most out of promises, here are some best practices and common pitfalls to avoid: Just like callback hell, promise chains can become nested and complicated. Use Promise.all() or async/await to keep your code clean and readable. Avoid Promise Hell: Unhandled promise rejections can cause hard-to-find bugs. Always use .catch() or try/catch with async/await. Always Catch Errors: Remember to return promises from your .then() handlers to keep the chain going. Chain Properly: Conclusion JavaScript promises are a robust tool for handling asynchronous operations, offering a more manageable approach to callbacks. By understanding and leveraging promises, you can write cleaner, more efficient JavaScript code. Keep experimenting with promises and async/await to find the patterns that work best for your applications. Also published . here

Mastering Asynchronous JavaScript: An In-Depth Guide to JavaScript Promises and Best Practices

When we discuss async EventHandlers, the first thing that comes to mind for many of us is that it’s the only exception that we seem to allow for . the dreaded async void setup When I had written about this before, I was excited that I was exploring a solution that involved actually allowing the async void to exist (without wanting to pull the rest of my hair out). For me, this was much more about some clever tricks we can use to overcome async EventHandlers than it was to provide solutions for avoiding the problem entirely. With that said though, there was a lot of traction on the article, which I am very thankful for, and some folks expressed opinions that they’d rather solve async EventHandlers a different way. I thought this was a great point, so I wanted to come up with an alternative approach that doesn’t fix async void, but it allows you to a-void it (see what I did there?) entirely while solving some of the challenges with async EventHandlers. In this article, I will present another solution that you can try out in your own code. We’ll address the pros and cons from my perspective with respect to how it can be used so you can decide if it makes sense for your use case. You can also find some interactable code on .NET fiddle right . Otherwise, you can if you’d like to clone it down locally to try it out. over here check the code out on GitHub A Companion Video! to check out the video! Click here The Problem The problem we face with async EventHandlers is that the signature for events that we can subscribe to in C# by default looks something like this: void TheObject_TheEvent(object sender, EventArgs e); And, you’ll notice that by having void out the front of this signature, we’re forced to use void in our own handlers in order to subscribe to the event. This means that if you want your handler to ever run async/await code, you’ll need to await inside your void method… Which introduces the big scary async void pattern that we’re told to avoid like the plague. And why? Because async void breaks the ability for exceptions to bubble up properly and can cause a ton of headaches as a result. addressed this by allowing us to get creative on the invocation side of things but… The previous article We might need support for this on objects we don’t control the invocation of events for (i.e., you are hooking up to a button’s click event in your favorite UI framework) Some people see the usage of the context inside of that solution as a hack (I’m not disagreeing with that either). … Specifically, with event handlers, we have some other more simple tricks we can do to support async EventHandlers! In my opinion, simple is better… so if you read my previous article on async void and your goal was really just to deal with EventHandlers, this should help. Solving Async EventHandlers With Try/Catch Based on the conditions previously stated, the exception handling breaks down over the boundary of the async void. If you have an exception that needs to bubble up crossing this boundary, then you’re going to be in for a fun time. And by fun, I mean if you enjoy debugging why stuff isn’t working and you don’t have a clear indication as to what’s breaking, then you’ll really have a great time. So what’s the easiest way to fix this? Let’s prevent exceptions from being able to cross this boundary in the first place by using a simple tool we have access to: try/catch. objectThatRaisesEvent.TheEvent += async (s, e) =>
{
    // if the try catch surrounds EVERYTHING in the handler, no exception can bubble up
    try
    {
        await SomeTaskYouWantToAwait();
    }
    catch (Exception ex)
    {
        // TODO: put your exception handling stuff here
    }

    // no exception can escape here if the try/catch surrounds the entire handler body
} As noted in the code above, if you place a try/catch block around the ENTIRE body of your event handler, then you can prevent any exceptions from bubbling up across that async void boundary. On the surface, it’s quite simple and doesn’t require anything fancy to implement this. Pros: Extremely simple. No complex mechanisms to understand. No packages required. You don’t need to be the owner of the class that raises the event for this to work. This means that this approach will work for all existing event-raising objects including WinForms and WPF UI components. Cons: You need to remember to do this… everywhere. It’s possible that as your code evolves over time, someone might accidentally write logic outside of the event handler’s try-catch that can throw exceptions With that said, this solution truly is simple, but I think we can do a little bit better. A (Slightly) Fancier Approach to Improving Async EventHandlers One improvement that I think we can make over the initially proposed solution is that we can make it a little bit more explicit that we have an async EventHandler that should be safe from bubbling up exceptions. This approach will also prevent code drift over time from causing problematic code from running outside of the event handler. However, it will not address the fact that you need to remember to add this in manually! Let’s check out the code: static class EventHandlers
{
    public static EventHandler<TArgs> TryAsync<TArgs>(
        Func<object, TArgs, Task> callback,
        Action<Exception> errorHandler)
        where TArgs : EventArgs
		=> TryAsync<TArgs>(
            callback,
            ex =>
            {
                errorHandler.Invoke(ex);
                return Task.CompletedTask;
            });
    
    public static EventHandler<TArgs> TryAsync<TArgs>(
        Func<object, TArgs, Task> callback,
        Func<Exception, Task> errorHandler)
        where TArgs : EventArgs
    {
        return new EventHandler<TArgs>(async (object s, TArgs e) =>
        {
	    try
            {
                await callback.Invoke(s, e);
            }
            catch (Exception ex)
            {
                await errorHandler.Invoke(ex);
            }
        });
    }
} The code above quite literally uses the exact same approach for preventing exceptions from crossing the async void boundary. We simply try to catch around the body of the event handler, but now we’ve bundled it up into an explicitly dedicated method to reuse. Here’s how it would look to apply it: someEventRaisingObject.TheEvent += EventHandlers.TryAsync<EventArgs>(
    async (s, e) =>
    {
        Console.WriteLine("Starting the event handler...");
    	await SomeTaskToAwait();
        Console.WriteLine("Event handler completed.");
    },
    ex => Console.WriteLine($"[TryAsync Error Callback] Our exception handler caught: {ex}")); We can see that we now have a delegate with an async Task signature to work with, and anything we put inside of that we rest assured will have a try/catch around it within the helper method we saw earlier. Here’s a screenshot showing the error handler callback properly capturing the exception: Pros: Still very simple. The wrapper function is *slightly* more complex, but still very basic. No packages required. You don’t need to be the owner of the class that raises the event for this to work. This means that this approach will work for all existing event-raising objects including WinForms and WPF UI components. The intention is more obvious for working with async EventHandlers because of the syntax when hooking up the handler to the event. Code drift that eventually throws more exceptions will still be wrapped inside the try/catch Cons: You still need to remember to hook this thing up! Closing Thoughts On Async EventHandlers While originally I set out to explore , the reader feedback was valid in that the examples focused on async EventHandlers, and there surely must be a more simple way. interesting ways to deal with async void In this article, we explored what I might argue is the most simple way to make your async EventHandlers behave properly, and the refined solution (in my opinion) only has the drawback that you need to remember to use it. A commenter had suggested that one could explore (AoP) to inject this sort of behavior across your application so that you wouldn’t need to go remember to do it. Aspect Oriented Programming There are some compile-time AoP frameworks that exist, but I’ll leave that as an exercise for you as the reader (because it’s also an exercise for me to go follow up on).

A Simple Safety Net For Async EventHandlers

Async ইভেন্টহ্যান্ডলারদের জন্য একটি সহজ নিরাপত্তা নেট

Ein einfaches Sicherheitsnetz für asynchrone EventHandler

Una red de seguridad simple para controladores de eventos asíncronos

Un filet de sécurité simple pour les gestionnaires d'événements asynchrones

Async EventHandlers के लिए एक साधारण सुरक्षा जाल

非同期 EventHandler の単純なセーフティ ネット

비동기 이벤트 핸들러를 위한 간단한 안전망

Uma rede de segurança simples para manipuladores de eventos assíncronos

Eşzamansız EventHandler'lar İçin Basit Bir Güvenlik Ağı

Mạng lưới an toàn đơn giản cho Trình xử lý sự kiện không đồng bộ

异步事件处理程序的简单安全网

What comes to a Python developer’s mind when he needs to write an I/O bound application? Of course . Why? Usually, because it’s a trend. And I’ve seen this case many times. But is this always a good idea? async Let’s consider a case when we’re writing a core I/O bound component but don’t need . async Let’s suppose we have several data sources, where data is represented by JSON messages. We have Kafka as a message broker, so data sources write messages to corresponding topics. And for each topic, we need a message processor to consume a message, find related records in reference tables, and save the message to the table with related foreign keys. It’s a typical case. So what do we do next? We think: “Hm, I need an I/O bound app, so I take ”. We don’t figure out the subject area, we don’t figure out requirements. We just take by default thinking that it’s optimal. But it’s not always optimal. Let’s try to take a better look. messages async async Let’s suppose we have PostgreSQL (and actually, on the real project we had that one), so we’ll need a proper tool for it. What choices do we have? The most popular and reliable are SQLAlchemy and Django ORM. If we need a powerful library, we choose SQLAlchemy. SQLAclehemy supports , but in a beta state. So we get a beta library in a core component in production. it doesn’t seem right. async If we take Django ORM, we get Django, less featured ORM, and support of by adapters. Not impressive either. async In our project, we didn’t have a high load and we didn’t need to process a big amount of messages per second. Moreover, sequential message processing was a strict requirement dictated by the subject area, and in this case, the asynchronous design is a fundamental mistake. So what do we have eventually? Wrong design that could cause serious bugs, “beta library” in production in a core component, and higher code complexity. I don’t think somebody would argue, that code is much simpler than . sync async But there are cases without such restrictions and with a higher load. What then? Is a good choice in these cases? Not always. could reduce the cost of infrastructure due to the better utilization of CPU time but it also increases the complexity of code, which in turn increases the cost of development and maintenance of a program and can cause more bugs. If the load is not too big and the system doesn’t need to be scaled so much, the company might take increasing the number of CPU cores as a solution. And that would be optimal in my opinion async async

When You Shouldn't Use Async in Python

Golang is a concurrent programming language. It has powerful features like and that can handle asynchronous tasks very well. Also, goroutines are not OS threads, and that's why you can spin up as many goroutines as you want without much overhead, it's stack size starts at only. So why ? Async/Await is a nice language feature that provides a simpler interface to asynchronous programming. Goroutines Channels 2KB async/await Project Link: https://github.com/Joker666/AsyncGoDemo How Does it Work? Started with F# and then C#, now in Python and Javascript, async/await is an extremely popular feature of a language. It simplifies the asynchronous method execution structure and, it reads like synchronous code. So much easier to follow for developers. Let's see a simple example in C# how async/await works Task Main(string[] args)
{
    Console.WriteLine( ); done = DoneAsync();
    Console.WriteLine( );
    Console.WriteLine( done);
} Task<int> DoneAsync()
{
    Console.WriteLine( ); Task.Delay( );
    Console.WriteLine( ); ;
} static async "Let's start ..." var "Done is running ..." await static async "Warming up ..." await 3000 "Done ..." return 1 We have the function that would be executed when the program is run. We have which is an async function. We stop the execution of the code with function for 3 seconds. Delay is an async function itself, so we call it with await. Main DoneAsync Delay await only blocks the code execution within the async function In the main function, we do not call with await. But the execution starts for . Only when we await it, we get the result back. The execution flow looks like this DoneAsync DoneAsync Let 's start ...
Warming up ...
Done is running ...
Done ...
1 This looks incredibly simple for asynchronous execution. Let's see how we can do it with Golang using Goroutines and Channels {
	r := ( )
	fmt.Println( ) {
		time.Sleep( * time.Second)
		r <- fmt.Println( )
	}() r
} {
	fmt.Println( )
	val := DoneAsync()
	fmt.Println( )
	fmt.Println(<- val)
} func DoneAsync () chan int make chan int "Warming up ..." go func () 3 1 "Done ..." return func main () "Let's start ..." "Done is running ..." Here, runs asynchronously and returns a channel. It writes a value to the channel once it's done executing the async task. In function, we invoke and keep doing our operations and then we read the value from the returned channel. It is a blocking call that waits till the value is written to the channel and after it gets the value it writes to the console. DoneAsync main DoneAsync Let 's start ...
Warming up ...
Done is running ...
Done ...
1 We see, we achieve the same outcome as the C# program but it doesn't look as elegant as async/await. While this is actually good, we are able to do a lot more granular things with this approach much easily, we can also implement async/await keywords in Golang with a simple struct and interface. Let's try that. Implementing Async/Await The full code is available in the project link. To implement async/await in Golang, we will start with a package directory named . The project structure looks like async .
├── async
│   └── async ├── main └── README.md .go .go In the async file, we write the simplest future interface that can handle async tasks. async Future {
	Await() {}
} future {
	await {}
} {} { f.await(context.Background())
} {}) Future { result {}
	c := ( {}) { (c)
		result = f()
	}() future{
		await: {} { { <-ctx.Done(): ctx.Err() <-c: result
			}
		},
	}
} package import "context" // Future interface has the method signature for await type interface interface type struct func (ctx context.Context) interface func (f future) Await () interface return // Exec executes the async function func Exec (f () func interface var interface make chan struct go func () defer close return func (ctx context.Context) interface select case return case return Not a lot is happening here, we add a interface that has the method signature. Next, we add a struct that holds one value, a function signature of the function. Now struct implements interface's Await method by invoking its own function. Future Await future await futute Future await Next in the function, we execute the passed function asynchronously in goroutine. And we return the function. It waits for the channel to close or context to read from. Based on whichever happens first, it either returns the error or the result which is an interface. Exec await Now armed with this new async package, let's see how we can change our current go code {
	fmt.Println( )
	time.Sleep( * time.Second)
	fmt.Println( ) } {
	fmt.Println( )
	future := async.Exec( {} { DoneAsync()
	})
	fmt.Println( )
	val := future.Await()
	fmt.Println(val)
} func DoneAsync () int "Warming up ..." 3 "Done ..." return 1 func main () "Let's start ..." func () interface return "Done is running ..." At the first glance, it looks much cleaner, we are not explicitly working with goroutine or channels here. Our function has been changed to a completely synchronous nature. In the main function, we use the package's method to handle . Which starts the execution of . The control flow is returned back to function which can execute other pieces of code. Finally, we make blocking call to and read back data. DoneAsync async Exec DoneAsync DoneAsync main Await Now the code looks much simpler and easier to read. We can modify our async package to incorporate a lot of other types of asynchronous tasks in Golang, but we would just stick to simple implementation for now in this tutorial. Conclusion We have gone through what async/await it and implemented a simple version of that in Golang. I would encourage you to look into async/await a lot more and see how it can ease the readability of the codebase much better.

Async/Await in Golang: An Introductory Guide

Async/Await en Golang: una guía introductoria

The text you are about to read describes an imaginary dialog between two programmers on their way to discover the of a library called . orgastic pleasures asyncFn asyncFn provides additional methods to eg. or to introduce “ ” for the promises returned by mock functions. This simplifies async unit testing by allowing tests that read chronologically, . jest.fn sinon.spy late resolve like a story From this, we get a of structuring and writing unit tests. new perspective Foreword Be warned that we’ll also cover topics such as , , , and “ ”. They are not the main topic here, but some words are spared for them nonetheless to justify . TDD pair programming evil pairing negation testing a way of thinking Best of luck, . Dear Reader Chapter 1: The premise and the foundation : My name is Fry, and I find it difficult to unit test async-stuff in JavaScript. Fry : Tell me more. Leela : Yup, say I wanted to implement something to this specification: Fry : As a player, I can play a game called Monster Beatdown : I can damage an encountered monster I encounter a monster I choose to attack it the monster loses hit points : I can try flee a monster I encounter a monster I choose to flee the monster eats me I lose : I can win the game by beating a monster until it is knocked out I encounter a monster I attack it until it has no hit points the monster is knocked out I win Feature Scenario Given And Then Scenario Given And Then And Scenario Given When Then And : The kicker here is that in this game, prompting the player for action is . Fry asynchronous : Mm-hmm, I think I get it. Let’s get our hands dirty and see where it lands us. Leela : Here’s an initial test for good measure: Fry gameWithoutDependencies ;

it( , () => { messagePlayerMock = jest.fn(); game = gameWithoutDependencies({ : messagePlayerMock,
  });

  game.encounterMonster();

  expect(messagePlayerMock).toHaveBeenCalledWith( ,
  );
}); import from './monsterBeatdown' 'when a monster is encountered, informs the player of this' const const messagePlayer 'You encounter a monster with 5 hit-points.' Contents of so far: ./monsterBeatdown.js ({ messagePlayer }) => ({ : {
    messagePlayer( );
  },
}); export default encounterMonster => () `You encounter a monster with 5 hit-points.` : Ok, so far so good. I see you chose to the mock-function for messaging the player as an argument for the function we are testing. Crystal. Full steam ahead. Leela inject : Right on. Next up will be my vision for asking the player if they want to attack, and if not, they lose. Fry Chapter 2: The problems emerge gameWithoutDependencies ;

it( , () => { messagePlayerMock = jest.fn(); game = gameWithoutDependencies({ : messagePlayerMock,
  }); game.encounterMonster();

  expect(messagePlayerMock).toHaveBeenCalledWith( ,
  );
});

it( , () => { askPlayerToHitMock = jest.fn(); game = gameWithoutDependencies({ : askPlayerToHitMock,
  }); game.encounterMonster();

  expect(askPlayerToHitMock).toHaveBeenCalledWith( );
});

it( , () => { askPlayerToHitMock = jest.fn( .resolve( )); messagePlayerMock = jest.fn(); game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
  }); game.encounterMonster();

  expect(messagePlayerMock).toHaveBeenCalledWith( ,
  );
});

it( , () => { askPlayerToHitMock = jest.fn( .resolve( )); messagePlayerMock = jest.fn(); game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
  }); game.encounterMonster();

  expect(messagePlayerMock).toHaveBeenCalledWith( );
}); import from './monsterBeatdown' 'when a monster is encountered, informs the player of this' async const const messagePlayer await 'You encounter a monster with 5 hit-points.' 'when a monster is encountered, asks player to attack' async const const askPlayerToHit await 'Do you want to attack it?' 'given a monster is encountered, when player chooses to flee, the monster eats the player' async // When asked to attack, choosing false means to flee. const => () Promise false const const askPlayerToHit messagePlayer await 'You chose to flee the monster, but the monster eats you in disappointment.' 'given a monster is encountered, when player chooses to flee, the game is over' async const => () Promise false const const askPlayerToHit messagePlayer await 'Game over.' Contents of so far: ./monsterBeatdown.js ({ messagePlayer = {}, askPlayerToHit = {} }) => ({ : { messagePlayer( );

    messagePlayer( );

    askPlayerToHit( );

    messagePlayer( ,
    );
  },
}); export default => () => () encounterMonster => () // For real life, the implementation below makes little sense. // This is intentional, and is so to prove a point later. // At this point, it suffices that all tests are green 'Game over.' 'You encounter a monster with 5 hit-points.' 'Do you want to attack it?' 'You chose to flee the monster, but the monster eats you in disappointment.' : Hold the phone. I see two problems here. One is the , and the other one is about the test describing occurrences in , making the test harder to read. Let’s start by fixing the first problem Leela duplication non-chronological order . Chapter 3: The clumsy fix : Uh, I was just about to remove the duplication anyway. , right? Fry Red-green-refactor : Right. Leela gameWithoutDependencies ;

describe( , () => { askPlayerToHitMock; messagePlayerMock; game;

  beforeEach( {
    askPlayerToHitMock = jest.fn();
    messagePlayerMock = jest.fn();

    game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
    });
  });

  it( , () => { game.encounterMonster();

    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => { game.encounterMonster();

    expect(askPlayerToHitMock).toHaveBeenCalledWith( ,
    );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.mockResolvedValue( ); game.encounterMonster();
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });
}); import from './monsterBeatdown' 'given a monster is encountered' let let let => () askPlayerToHit messagePlayer 'informs the player of this' async await 'You encounter a monster with 5 hit-points.' 'asks player to attack' async await 'Do you want to attack it?' 'when player chooses to flee' async // When asked to attack, choosing false means to flee. false await 'player is informed of the grim outcome' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is over' 'Game over.' Contents of so far: ./monsterBeatdown.js // no changes, as only tests were refactored. : There. Now the duplication for the test setup has been removed, but only to a degree. However, the other problem of test not being written in chronological order prevents us from removing all the duplication. See how we are forced to repeat because needs to know how to behave before it is called. Worse yet, this makes the “describe” dishonest, as it claims to display how " ", but in reality, this is something that happens only later in the test setup, if ever. All this kind of bums me out, and I’ve felt the pain of this getting out of hand as in real-life requirements and features start to pile up. I know many ways to ease the pain a little bit, but nothing I’ve tried has left me comfortable. Fry game.encounterMonster() askPlayerToHitMock given a monster is encountered : I see. Let’s now introduce as an expansion to the normal to tackle all. Behold. Leela asyncFn jest.fn Chapter 4: The beholding of asyncFn gameWithoutDependencies ; asyncFn ;

describe( , () => { askPlayerToHitMock; messagePlayerMock; game;

  beforeEach( { askPlayerToHitMock = asyncFn();
    messagePlayerMock = jest.fn();

    game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
    });

    game.encounterMonster();
  });

  it( , () => {
    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => { expect(askPlayerToHitMock).toHaveBeenCalledWith( ,
    );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });
}); import from './monsterBeatdown' import from '@asyncFn/jest' 'given a monster is encountered' let let let => () // Note: Before we used jest.fn here instead of asyncFn. askPlayerToHit messagePlayer 'informs the player of this' async 'You encounter a monster with 5 hit-points.' 'asks player to attack' // Note how even if askPlayerToHitMock is now a mock made using asyncFn(), it still is a jest.fn(). This means eg. toHaveBeenCalledWith can be used with it. 'Do you want to attack it?' 'when player chooses to flee' async // Note how choosing false here still means fleeing combat, but here we use asyncFn's .resolve() to control the outcome. And then we await for the consequences of that. await false 'player is informed of the grim outcome' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is over' 'Game over.' Contents of so far: ./monsterBeatdown.js // no changes, as only tests were refactored. : Now the duplication is gone, and everything takes place in clean, chronological order. It kind of reads like a story, don’t you think? Leela : Mm-hmm. I see it. I have a good feeling about it. But something is bothering me with the production code. I see that the tests are all green, but clearly, the code does not do anything sensical. It just blows through, merely satisfying the tests. Fry : You got me there. We are, in fact doing something called “evil pairing”. If you want to mold the production code your way, you need to order it from by writing a test first. Let me show you how. Leela the universe very specific Chapter 5: Evil is good and negative is positive gameWithoutDependencies ; asyncFn ;

describe( , () => { askPlayerToHitMock; messagePlayerMock; game;

  beforeEach( {
    askPlayerToHitMock = asyncFn();
    messagePlayerMock = jest.fn();

    game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
    });

    game.encounterMonster();
  });

  it( , () => {
    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => {
    expect(askPlayerToHitMock).toHaveBeenCalledWith( ,
    );
  }); it( , () => {
    expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
    );
  }); it( , () => {
    expect(messagePlayerMock).not.toHaveBeenCalledWith( );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( );
    }); it( , () => {
      expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });
}); import from './monsterBeatdown' import from '@asyncFn/jest' 'given a monster is encountered' let let let => () askPlayerToHit messagePlayer 'informs the player of this' async 'You encounter a monster with 5 hit-points.' 'asks player to attack' 'Do you want to attack it?' // This is the "negation test" referred later in narrative. 'when player has not chosen anything yet, the player is not eaten' 'You chose to flee the monster, but the monster eats you in disappointment.' // This is another negation test. 'when player has not chosen anything yet, the game is not over' 'Game over.' 'when player chooses to flee' async await false 'player is informed of the grim outcome' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is over' 'Game over.' 'when player chooses to attack' async await true // Another negation test 'player does not lose' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is over' 'Game over.' Contents of so far: ./monsterBeatdown.js ({ messagePlayer, askPlayerToHit }) => ({ : () => {
    messagePlayer( ); playerChoseToAttack = askPlayerToHit( ,
    ); (!playerChoseToAttack) {
      messagePlayer( ,
      );
    }

    messagePlayer( );
  },
}); export default encounterMonster async 'You encounter a monster with 5 hit-points.' const await 'Do you want to attack it?' if 'You chose to flee the monster, but the monster eats you in disappointment.' 'Game over.' : There. I’ve reduced the “evilness” of the code by adding something we call “negation tests”. By writing tests that describe what is not supposed to happen we forced the production code to make a little bit more sense. And if you are wondering where the name “evil pairing” comes from, it comes from playing unit testing ping-pong in such a way that the production code is always written in the most evil way possible, ie. the code is non-sensical for real-life, yet it still satisfies the tests. The only way out of this is to force sense in production through unit tests. Sometimes you can tell coders are evil pairing from the . In general, this “ ” helps produce code that is very robust for the sake of refactoring, and also allows programmers to hone their a little bit more. But I digress. However important this may be, it is slightly off-course. What is relevant is that supports evil pairing as a line of thinking. Let’s motor on. Leela evil laughter evil pairing -mentality TDD-mojo asyncFn Chapter 6: The multiplicity challenge and the enlightenment : For sure. Can we see how the game develops some steps further? Particularly, I’ve been suffering from testing of functions that are called , yet return promises. Fry multiple times : I feel you. My guess is an asyncFn-mock will get called multiple times soon enough, when we start attacking the monster multiple times. Let’s see how that pans out. Leela gameWithoutDependencies ; asyncFn ;

describe( , () => { askPlayerToHitMock; messagePlayerMock; game;

  beforeEach( {
    askPlayerToHitMock = asyncFn();
    messagePlayerMock = jest.fn();

    game = gameWithoutDependencies({ : askPlayerToHitMock, : messagePlayerMock,
    });

    game.encounterMonster();
  });

  it( , () => {
    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => {
    expect(askPlayerToHitMock).toHaveBeenCalledWith( ,
    );
  });

  it( , () => {
    expect(messagePlayerMock).not.toHaveBeenCalledWith( );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });
  });

  it( , () => { askPlayerToHitMock.resolve( );

    expect(messagePlayerMock).toHaveBeenCalledWith( ,
    );
  });

  describe( , () => {
    beforeEach( () => { askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( );
    });

    it( , () => {
      expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( ,
      );
    });

    it( , () => {
      expect(messagePlayerMock).toHaveBeenCalledWith( );
    });

    it( , () => {
      expect(messagePlayerMock).not.toHaveBeenCalledWith( ,
      );
    });
  });

  it( , () => { askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( ); askPlayerToHitMock.resolve( );

    expect(messagePlayerMock).not.toHaveBeenCalledWith( );
  });
}); import from './monsterBeatdown' import from '@asyncFn/jest' 'given a monster is encountered' let let let => () askPlayerToHit messagePlayer 'informs the player of this' async 'You encounter a monster with 5 hit-points.' 'asks player to attack' 'Do you want to attack it?' 'when player has not chosen anything yet, the game is not lost' 'Game over.' 'when player chooses to flee' async await false 'player is informed of the grim outcome' 'You chose to flee the monster, but the monster eats you in disappointment.' 'the game is not won' 'You knock out the monster. You are a winner.' 'the game is over' 'Game over.' 'when player hits the monster, the monster loses a hit-point' async await true 'You hit the monster. It now has 4 hit-points remaining.' 'when player hits the monster enough times to knock it out' async // Here the monster is attacked multiple times, and therefore, askPlayerToHitMock is called multiple times, and then the test resolves the mock multiple times. // Using asyncFn, the logic remains the same: things that happen are written and observed in chronological order. await true await true await true await true await true 'does not show out-of-place message about no hit-points' 'You hit the monster. It now has 0 hit-points remaining.' 'player is congratulated' 'You knock out the monster. You are a winner.' 'game is won' 'Game over.' 'the game is not lost' 'You chose to flee the monster, but the monster eats you in disappointment.' 'when player hits the monster until it only has 1 hit-point remaining, the game is not over yet' async // Hit the monster only 4 times instead of 5. await true await true await true await true 'Game over.' The final form of : ./monsterBeatdown.js ({ askPlayerToHit, messagePlayer }) => ({ : () => { monsterHitPoints = ;

    messagePlayer( ,
    ); ( askPlayerToHit( )) {
      monsterHitPoints--; (monsterHitPoints === ) { ;
      }

      messagePlayer( ,
      );
    } (monsterHitPoints === ) {
      messagePlayer( );
    } {
      messagePlayer( ,
      );
    }

    messagePlayer( );
  },
}); export default encounterMonster async let 5 `You encounter a monster with hit-points.` ${monsterHitPoints} while await 'Do you want to attack it?' if 0 break `You hit the monster. It now has hit-points remaining.` ${monsterHitPoints} if 0 'You knock out the monster. You are a winner.' else 'You chose to flee the monster, but the monster eats you in disappointment.' 'Game over.' : There. Now the monster gets hit multiple times, all while things still happen in a clear order. Our work is done here. Leela : Color me enlightened. This has changed my view of the world as a programmer and a human being. I shall make sacrifices in your honor and get an asyncFn-tattoo. My grandchildren will know your name. Fry Summary Having witnessed , we’ve seen how the of permitted troublesome tests, with awkward readability, to be refactored orderly as a story, with significantly better readability. the adventure late resolve asyncFn is something that we, as a team, have used for multiple years now. It has categorically changed the way we approach unit testing of asynchronous code in JavaScript. Which we so adore. asyncFn See more about asyncFn for or . jest sinon Who are we? asyncFn is lovingly crafted by at from . Your Pals Team: Igniter Houston Inc. Consulting We are a software development team of friends, with proven tradition in professional excellence. We specialize in holistic without sacrificing . rapid deployments quality Come say hi at , us, or check out the . We just might be available for hiring ;) Gitter email team itself Previously published at https://medium.com/houston-io/how-to-unit-test-asynchronous-code-for-javascript-in-2020-41c124be2552

Testing Asynchronous JS Code: 2020 Edition

This article was originally posted on my blog. A few weeks ago, I was working on a Python script to extract books' metadata for a . After a couple of hours, I realized that I needed to make thousands of requests to the Google Books API to get the data. So I thought there had to be a way of speeding up the process. content-based recommender As I enjoy learning, especially when it's also a chance of procrastinating on my goals, I decided to build a project using . Afterward, feeling guilty for the time wasted, I decided to write this tutorial with what I learned in the process. asyncio This article aims to provide the basics of how to use for making asynchronous requests to an API. I focus mostly on the actual code and skip most of the theory (besides the short introduction below). However, if you are looking for a more in-depth introduction to , check the recommendations in the references[1]. asyncio asyncio in 30 Seconds or Less asyncio is a Python library that allows you to execute some tasks in a [2] manner. It is commonly used in web-servers and database connections. It is also useful for speeding up IO-bound tasks, like services that require making many requests or do lots of waiting for external APIs. asyncio seemingly concurrent The essence of is that it allows the program to continue executing other instructions while waiting for specific processes to finish (e.g., a request to an API). In this tutorial, we will see how to use for accelerating a program that makes multiple requests to an API. asyncio asyncio Sequential vs. Asynchronous So let's get down to business. To get the most out of this tutorial, try running the code yourself. You can copy-and-paste the code in a Jupyter Notebook and run it without modifications. Just remember to first install the required libraries ( and .) requests aiohttp We'll build a sequential and an asynchronous version of a small program and compare their results and structure. Both programs will do the same: Read a list of ISBNs (international identifier of books) Request the books' metadata to the Google Books API Parse the results from the requests Print the results to the screen The algorithm would look something like the diagram below. We'll now compare two possible approaches for building this algorithm. First, , which executes the requests sequentially. Then, , which uses to run requests asynchronously. Option A Option B asyncio Option A: Sequential Algorithm A sequential version of that algorithm could look as follows: os requests requests.exceptions HTTPError


GOOGLE_BOOKS_URL = LIST_ISBN = [ , , , , , , , , , ,
] volume_info = item.get( , {})
    title = volume_info.get( , )
    subtitle = volume_info.get( , )
    description = volume_info.get( , )
    published_date = volume_info.get( , ) (
        title,
        subtitle,
        description,
        published_date,
    ) url = GOOGLE_BOOKS_URL + isbn
    response = :
        response = session.get(url)
        response.raise_for_status()
        print( ) HTTPError http_err:
        print( ) Exception err:
        print( )
    response_json = response.json()
    items = response_json.get( , [{}])[ ] items requests.Session() session: isbn LIST_ISBN: :
            response = get_book_details_seq(isbn, session)
            parsed_response = extract_fields_from_response(response)
            print( ) Exception err:
            print( ) import import from import "https://www.googleapis.com/books/v1/volumes?q=isbn:" '9780002005883' '9780002238304' '9780002261982' '9780006163831' '9780006178736' '9780006280897' '9780006280934' '9780006353287' '9780006380832' '9780006470229' : def extract_fields_from_response (item) """Extract fields from API's response""" "volumeInfo" "title" None "subtitle" None "description" None "publishedDate" None return : def get_book_details_seq (isbn, session) """Get book details using Google Books API (sequentially)""" None try f"Response status ( ): " {url} {response.status_code} except as f"HTTP error occurred: " {http_err} except as f"An error ocurred: " {err} "items" 0 return with as for in try f"Response: " {json.dumps(parsed_response, indent= )} 2 except as f"Exception occured: " {err} pass Now, let's breakdown the code to understand what's going on. As usual, we start by importing the required libraries. Then, we define two variables: for specifying the URL of the Google's API we'll use for the requests. GOOGLE_BOOKS_URL which is a sample list of ISBNs for testing the program. LIST_ISBN , For illustration purposes, this how a request to the Google Books API looks like: https://www.googleapis.com/books/v1/volumes?q=isbn:9780002005883 Next, we define the function. This function takes as input the response from the API and extracts the fields we're interested in. extract_fields_from_response The parsing process in is based on the response's structure from the Google Books API, which looks as follows: extract_fields_from_response { : , : , : [
  { : , : , : , : , : { : , : ,
    ... "kind" "books#volumes" "totalItems" 1 "items" "kind" "books#volume" "id" "3Mx4QgAACAAJ" "etag" "FWJF/JY16xg" "selfLink" "https://www.googleapis.com/books/v1/volumes/3Mx4QgAACAAJ" "volumeInfo" "title" "Mapping the Big Picture" "subtitle" "Integrating Curriculum and Assessment, K-12" Finally, we go into the most relevant parts of the program: how we make requests to the Google Books API. url = GOOGLE_BOOKS_URL + isbn
    response = :
        response = session.get(url)
        response.raise_for_status()
        print( ) HTTPError http_err:
        print( ) Exception err:
        print( )
    response_json = response.json()
    items = response_json.get( , [{}])[ ] items requests.Session() session: isbn LIST_ISBN: :
            response = get_book_details_seq(isbn, session)
            parsed_response = extract_fields_from_response(response)
            print( ) Exception err:
            print( ) : def get_book_details_seq (isbn, session) """Get book details using Google Books API (sequentially)""" None try f"Response status ( ): " {url} {response.status_code} except as f"HTTP error occurred: " {http_err} except as f"An error ocurred: " {err} "items" 0 return with as for in try f"Response: " {json.dumps(parsed_response, indent= )} 2 except as f"Exception occured: " {err} pass There are two major pieces here: , which is the function that executes the requests. It takes as input an ISBN and a session object[4] and returns the response from the API as a JSON structure. It also handles possible errors, like providing a wrong URL or going over your daily quota of requests. get_book_details_seq The code block under , is where the actual execution of requests happens. It iterates through the list of ISBNs, getting the book details, parsing them, and finally printing them to the screen. with requests.Session() as session For me, . If you only need to do this a couple of times, you will not find much benefit from using . However, if instead of 10 requests, you need to do 10,000, having some concurrency in your program pays out. In the next section, we'll see how to make this algorithm faster using . executing this process takes ranges from 4 to 6 seconds asyncio asyncio Option B: Asynchronous Algorithm An asynchronous version of the same algorithm may look something as follows: aiohttp asyncio os aiohttp ClientSession


GOOGLE_BOOKS_URL = LIST_ISBN = [ , , , , , , , , , ,
] item = response.get( , [{}])[ ]
    volume_info = item.get( , {})
    title = volume_info.get( , )
    subtitle = volume_info.get( , )
    description = volume_info.get( , )
    published_date = volume_info.get( , ) (
        title,
        subtitle,
        description,
        published_date,
    ) url = GOOGLE_BOOKS_URL + isbn :
        response = session.request(method= , url=url)
        response.raise_for_status()
        print( ) HTTPError http_err:
        print( ) Exception err:
        print( )
    response_json = response.json() response_json :
        response = get_book_details_async(isbn, session)
        parsed_response = extract_fields_from_response(response)
        print( ) Exception err:
        print( ) ClientSession() session: asyncio.gather(*[run_program(isbn, session) isbn LIST_ISBN]) import import import from import "https://www.googleapis.com/books/v1/volumes?q=isbn:" '9780002005883' '9780002238304' '9780002261982' '9780006163831' '9780006178736' '9780006280897' '9780006280934' '9780006353287' '9780006380832' '9780006470229' : def extract_fields_from_response (response) """Extract fields from API's response""" "items" 0 "volumeInfo" "title" None "subtitle" None "description" None "publishedDate" None return async : def get_book_details_async (isbn, session) """Get book details using Google Books API (asynchronously)""" try await 'GET' f"Response status ( ): " {url} {response.status} except as f"HTTP error occurred: " {http_err} except as f"An error ocurred: " {err} await return async : def run_program (isbn, session) """Wrapper for running program in an asynchronous manner""" try await f"Response: " {json.dumps(parsed_response, indent= )} 2 except as f"Exception occured: " {err} pass async with as await for in First, check the function. An keyword prepends it. This keyword tells Python that your function is a . Then, in the function's body, there are two await keywords. These tell that to suspend execution and give back control to the event loop, while the operation it is awaiting finishes. get_book_details_async async coroutine coroutine A is a type of generator function in Python that, instead of producing values, consumes values[4]. The interesting thing about it is that its execution pauses while waiting for new data being sent to it. In our case, this allows the execution of other parts of the program to continue in a manner. coroutine seemingly concurrent In this case, the execution of is suspended while the request is being performed: get_book_details_async . await session.request(method='GET', url=url) It is suspended again, while the request response is being parsed into a JSON structure: . await response.json() Next, we have the coroutine. This one is simply a wrapper around the pipeline of getting a response from the API, parsing it, and printing the results in the screen. It the execution of the coroutine. run_program awaits get_book_details_async Finally, we have the code block under . Using the syntax, we tell the program to schedule all the tasks based on the list of coroutines we provided. This is what allows us to execute tasks concurrently. async with ClientSession() as session: asyncio.gather For me, running this process takes around 800-1000 milliseconds. Results Comparing both versions, we see that than the sequential version. If we increase the number of requests, you'll likely get an even higher speedup. Besides, , which makes using asyncio an excellent option for the kind of task we reviewed in the tutorial. the asynchronous one is around 4 to 7.5 times faster the version using asyncio is not much more complicated than the sequential version Additional recommendations Here are some tips I gathered while working with asyncio: keeps changing all the time, so be wary of old Stack Overflow answers. Many of them are not up to date with the current best practices asyncio APIs will not allow you to run unlimited concurrent requests. To overcome that, take a look at . It will enable you to limit the concurrency of your application. asyncio's Semaphore Not all programs can be speedup with asyncio. Research the type of issue you are facing before doing any substantial modification of code. Other alternatives might work for you (e.g., ) threading, multiprocessing I made a complete version of the program we went through in this tutorial for getting the metadata of almost 7 thousand books. Here's a link to it: . Google Books Crawler Notes and References [1] Real Python has a two of amazing articles introducing : and asyncio Async IO in Python Speed Up Your Python Program With Concurrency [2] It is not strictly concurrent execution. But in practical terms, it looks like it is. [3] S. Buczyński, (2017) What Is the use case of coroutines and asyncio in Python 3.6? [4] The session object is a functionality from the requests library that allows you to persist certain parameters across sessions. This usually results in requests with lower latency. Read more . here [5] D. Beasly, (2009) A Curious Course on Couroutines and Concurrency

Fast & Asynchronous: Accelerate Your Requests Using Python's asyncio

Recently at my workplace our IT team finally upgraded our distributed Python versions to 3.5.0. While this is a huge upgrade from 2.6, this still came with some growing pains. Unfortunately, Python 3.5.0 doesn’t meet some of the minimum requirements of some popular libraries, including . aiohttp With these restrictions I still needed to write a script that could pull hundreds of .csv files from our APIs and manipulate the data. Python by itself isn’t event-driven and natively asynchronous (like NodeJS), but the same effect can still be achieved. This article will help detail what I learned while also showing the benefits of asynchronous operations. If you have a higher version of Python available (3.5.2+), I highly recommend using aiohttp instead. It’s an incredibly robust library and a great solution for this kind of problem. There are many tutorials online detailing how best to use the library. Disclaimer: Assumptions This article makes the following assumptions: You already have familiarity with Python and most of its syntaxYou already have familiarity with basic web requestsYou have a lose concept of asynchronous operations If you’re just looking for the solution, scroll down to the bottom and the full code is posted. Enjoy! Setup Before getting started, ensure that you have installed on your machine. The easiest way to install is by typing the following command into your terminal: requests $ python -m pip install requests Alternatively, if you don’t have administrative permissions you can install the library with this command: $ python -m pip install requests --user The wrong approach: synchronous requests To demonstrate the benefits of our parallel approach, let’s first look at approaching the problem in a synchronous manner. I’ll also give an overview of what’s going on in the code. Ultimately, we want to do able to perform a GET request to the URL containing the .csv file and measure the time it takes to read the text inside. We’ll be downloading multiple .csv files of varying sizes from /, which provides plenty of data for our example. https://people.sc.fsu.edu/~jburkardt/data/csv As a disclaimer, we’ll be using the Session object from the requests library to perform our GET request. First, we’ll need a function that executes the web request:c base_url = session.get(base_url + csv) response:
        data = response.text response.status_code != :
            print( .format(url)) data : def fetch (session, csv) "https://people.sc.fsu.edu/~jburkardt/data/csv/" with as if 200 "FAILURE::{0}" # Return .csv data for future consumption return View code gist here This function takes in a object and the name of the .csv file desired, performs the web request, then returns the text inside the response. Session Next, we need a function that can efficiently loop a list of our desired files and measure the time it takes to perform the request: timeit default_timer() csvs_to_fetch = [ , , , , , , , , , , , ] requests.Session() session:
        print( .format( , )) total_start_time = default_timer() csv csvs_to_fetch:
            fetch(session, csv)
            elapsed = default_timer() - total_start_time
            time_completed_at = .format(elapsed)
            print( .format(csv, time_completed_at)) from import : def get_data_synchronous () "ford_escort.csv" "cities.csv" "hw_25000.csv" "mlb_teams_2012.csv" "nile.csv" "homes.csv" "hooke.csv" "lead_shot.csv" "news_decline.csv" "snakes_count_10000.csv" "trees.csv" "zillow.csv" with as "{0:<30} {1:>20}" "File" "Completed at" # Set any session parameters here before calling `fetch` # For instance, if you needed to set Headers or Authentication # this can be done before starting the loop for in "{:5.2f}s" "{0:<30} {1:>20}" View code gist here This function creates our object and then loops through each .csv file in the list. Once the operation is completed, the measured time is calculated and displayed in an easy-to-read format. Session csvs_to_fetch fetch Finally, our function will be simple (for now) and call our function: main get_data_synchronous()

main() : def main () # Simple for now View code gist here Once we put it all together, here is what the code looks like for our synchronous example: requests timeit default_timer base_url = session.get(base_url + csv) response:
        data = response.text response.status_code != :
            print( .format(url)) data csvs_to_fetch = [ , , , , , , , , , , , ] requests.Session() session:
        print( .format( , )) total_start_time = default_timer() csv csvs_to_fetch:
            fetch(session, csv)
            elapsed = default_timer() - total_start_time
            time_completed_at = .format(elapsed)
            print( .format(csv, time_completed_at)) get_data_synchronous()

main() import from import : def fetch (session, csv) "https://people.sc.fsu.edu/~jburkardt/data/csv/" with as if 200 "FAILURE::{0}" # Return .csv data for future consumption return : def get_data_synchronous () "ford_escort.csv" "cities.csv" "hw_25000.csv" "mlb_teams_2012.csv" "nile.csv" "homes.csv" "hooke.csv" "lead_shot.csv" "news_decline.csv" "snakes_count_10000.csv" "trees.csv" "zillow.csv" with as "{0:<30} {1:>20}" "File" "Completed at" # Set any session parameters here before calling `fetch` # For instance, if you needed to set Headers or Authentication # this can be done before starting the loop for in "{:5.2f}s" "{0:<30} {1:>20}" : def main () # Simple for now View code gist here Let’s take a look at the results when we run this script: Synchronous example. Notice how each operation doesn’t start until the last one is completed Thankfully, we can vastly improve this performance with Python 3’s built-in library! asyncio The right approach: performing multiple requests at once asynchronously In order to get this to work, we’ll have to rework some of our existing functions. Beginning with : fetch requests timeit default_timer START_TIME = default_timer() base_url = session.get(base_url + csv) response:
        data = response.text response.status_code != :
            print( .format(url)) elapsed = default_timer() - START_TIME
        time_completed_at = .format(elapsed)
        print( .format(csv, time_completed_at)) data import from import # We'll need access to this variable later : def fetch (session, csv) "https://people.sc.fsu.edu/~jburkardt/data/csv/" with as if 200 "FAILURE::{0}" # Now we will print how long it took to complete the operation from the # `fetch` function itself "{:5.2f}s" "{0:<30} {1:>20}" return View code gist here Next, we need to make our function asynchronous: get_data asyncio timeit default_timer concurrent.futures ThreadPoolExecutor csvs_to_fetch = [ , , , , , , , , , , , ]
    print( .format( , )) ThreadPoolExecutor(max_workers= ) executor: requests.Session() session: loop = asyncio.get_event_loop() START_TIME = default_timer() tasks = [
                loop.run_in_executor(
                    executor,
                    fetch,
                    *(session, csv) ) csv csvs_to_fetch
            ] response asyncio.gather(*tasks): import from import from import async : def get_data_asynchronous () "ford_escort.csv" "cities.csv" "hw_25000.csv" "mlb_teams_2012.csv" "nile.csv" "homes.csv" "hooke.csv" "lead_shot.csv" "news_decline.csv" "snakes_count_10000.csv" "trees.csv" "zillow.csv" "{0:<30} {1:>20}" "File" "Completed at" # max_workers is set to 10 simply for this example, Note: # you'll have to tweak with this number for your own projects # as you see fit with 10 as with as # Set any session parameters here before calling `fetch` # Initialize the event loop # Set the START_TIME for the `fetch` function # Use list comprehension to create a list of # tasks to complete. The executor will run the `fetch` # function for each csv in the csvs_to_fetch list # Allows us to pass in multiple arguments to `fetch` for in # Initializes the tasks to run and awaits their results for in await pass View code gist here This code will now create multiple threads for each .csv file and execute the function for each that needs to be downloaded. fetch Finally, our function needs a small tweak to properly initialize our async function: main loop = asyncio.get_event_loop()
    future = asyncio.ensure_future(get_data_asynchronous())
    loop.run_until_complete(future)

main() : def main () View code gist here Now, let’s run the new code and see the results: Asynchronous example. Notice how the files are not being obtained in order. With this small change, all 12 of these .csv files were able to be downloaded in 3.43s vs 10.84s. That is a nearly 70% decrease in the time it took to download! The Asynchronous Code requests asyncio concurrent.futures ThreadPoolExecutor timeit default_timer

START_TIME = default_timer() base_url = session.get(base_url + csv) response:
        data = response.text response.status_code != :
            print( .format(url))

        elapsed = default_timer() - START_TIME
        time_completed_at = .format(elapsed)
        print( .format(csv, time_completed_at)) data csvs_to_fetch = [ , , , , , , , , , , , ]
    print( .format( , )) ThreadPoolExecutor(max_workers= ) executor: requests.Session() session: loop = asyncio.get_event_loop()
            START_TIME = default_timer()
            tasks = [
                loop.run_in_executor(
                    executor,
                    fetch,
                    *(session, csv) ) csv csvs_to_fetch
            ] response asyncio.gather(*tasks): loop = asyncio.get_event_loop()
    future = asyncio.ensure_future(get_data_asynchronous())
    loop.run_until_complete(future)

main() import import from import from import : def fetch (session, csv) "https://people.sc.fsu.edu/~jburkardt/data/csv/" with as if 200 "FAILURE::{0}" "{:5.2f}s" "{0:<30} {1:>20}" return async : def get_data_asynchronous () "ford_escort.csv" "cities.csv" "hw_25000.csv" "mlb_teams_2012.csv" "nile.csv" "homes.csv" "hooke.csv" "lead_shot.csv" "news_decline.csv" "snakes_count_10000.csv" "trees.csv" "zillow.csv" "{0:<30} {1:>20}" "File" "Completed at" with 10 as with as # Set any session parameters here before calling `fetch` # Allows us to pass in multiple arguments to `fetch` for in for in await pass : def main () View code gist here I hope you enjoyed this article and can use these skills for any projects that require an older Python version (or maybe without as many dependencies). Although Python may not have a straightforward to an async / await pattern, it isn’t difficult at all to achieve fantastic results. Enjoy!

How to run asynchronous web requests in parallel with Python 3.5 (without aiohttp)

You like asynchronous programming, and appreciate the elegance of await, tasks and similar  in making asynchronous programming more intuitive and understandable and while all is good, you will likely find that async debugging  is quite tough! Exceptions in the async processes often won't show up in the console or logs until the program itself is terminated.  You end up watching your program just wait when it should have shown an exception and spurred you into debugging! The stages of asynchronous creativity are as below: The underlying reason for the delayed logging of exceptions  is that they bubble up to the calling process, which itself is an async process and will process the exceptions only when it is ready to run in the async loop. Often the calling process is waiting on an 'await' and hence ends up not processing any exceptions, resulting in situation depicted above! 'Whats happening? When the what happened?' Before delving into the issue, here is a solution (you do not have to read any further if you already know the problem and use the decorator below as befitting your code): asyncio pprint, traceback, functools datetime print( ) : func(*args) Exception E:
                print( ) wrapped deco import import import : def exceptionCatcherForAsyncDecorator () : def deco (func) @functools.wraps(func) async : def wrapped (*args) 'wrap function invoked' try return await except as f'Exception occured at: \n ' {datetime.datetime.now()} {pprint.pformat(traceback.format_exc())} raise #re-raise exception to allow process in calling function return return This is a decorator for the async functions that logs exceptions as they occur and hence the problem shows up in your logs immediately as they occur. In this decorator the print function is used to log the exceptions. Problem Description I got into this issue when coding a simple async program containing two async process, one generating data for the other and each executing indefinitely (or forever -- limited to 30 seconds in the demo below). Here is the abstracted code simulating the two async processes also showing the decorator at the top: asyncio pprint, traceback, functools datetime print( ) : func(*args) Exception E:
                print( ) wrapped deco random asyncio.sleep( )
    print( ) m/n :
        myNumerator=random.randint( , )
        myDenominator=random.randint( , )
        result= doProcessing(myNumerator,myDenominator)
        print( ) evtCreator_task = asyncio.create_task(callProcessing()) sleepTime= print( ) asyncio.sleep(sleepTime) ( )
    print( )

asyncio.run(main()) import import import : def exceptionCatcherForAsyncDecorator () : def deco (func) @functools.wraps(func) async : def wrapped (*args) 'wrap function invoked' try return await except as f'Exception occured at: \n ' {datetime.datetime.now()} {pprint.pformat(traceback.format_exc())} raise #re-raise exception to allow process in calling function return return ########## Main program starts here ############## import #@exceptionCatcherForAsyncDecorator() -- decorator commented out for demo async : def doProcessing (m,n) await 1 f' ; ; ' {m} {n} {m/n} return async : def callProcessing () while True 1 100 0 2 await f'result = ' {result} async : def main () # spawn task to create events # Sleep for and terminate. 30 f'Time now: - Sleeping for seconds..' {datetime.datetime.now()} {sleepTime} await print f'Time now: ' {datetime.datetime.now()} '..Terminated!' The two processes in the main program are: a.  doProcessing (simulates complex processing). Here it  'occasionally' generates an exception 'division by 0'. b. callProcessing (supplies input to doProcessing) The third process main() starts the program.  The main() terminates after 30 seconds and causes the doProcessing and callProcessing to be cancelled. Running this code without the decorator (as shown above, the decorator is commented out) results in the exception being visible only after 30 seconds as shown below: $ python asyncDecorator.py 
Time now: 2019-09-30 11:24:13.785183 - Sleeping for 30 seconds..
Time now: 2019-09-30 11:24:43.799021  <== 30 seconds have elapsed..
..Terminated!
Task exception was never retrieved
future: <Task finished coro=<callProcessing() done, defined at asyncDecorator.py:26> exception=ZeroDivisionError('division by zero')>
Traceback (most recent call last):
  File "asyncDecorator.py", line 30, in callProcessing
    result=await doProcessing(myNumerator,myDenominator)
  File "asyncDecorator.py", line 23, in doProcessing
    print(f'{m};{n};{m/n}')
ZeroDivisionError: division by zero Running the same code with the decorator active (i.e. decorator uncommented) results in the exception being logged immediately as it occurred: $ python asyncDecorator.py 
Time now: 2019-09-30 11:32:29.895473 - Sleeping for 30 seconds..
wrap function invoked
Exception occured at: 2019-09-30 11:32:30.896903  <== exception logged as it occurs !
 ('Traceback (most recent call last):\n'
 '  File "asyncDecorator.py", line 11, in wrapped\n'
 '    return await func(*args)\n'
 '  File "asyncDecorator.py", line 23, in doProcessing\n'
 "    print(f'{m};{n};{m/n}')\n"
 'ZeroDivisionError: division by zero\n')
Time now: 2019-09-30 11:32:59.913337 <== 30 seconds elapsed..
..Terminated!
Task exception was never retrieved <== exception logged again due to re-raise by decorator
future: <Task finished coro=<callProcessing() done, defined at asyncDecorator.py:26> exception=ZeroDivisionError('division by zero')>
Traceback (most recent call last):
  File "asyncDecorator.py", line 30, in callProcessing
    result=await doProcessing(myNumerator,myDenominator)
  File "asyncDecorator.py", line 11, in wrapped
    return await func(*args)
  File "asyncDecorator.py", line 23, in doProcessing
    print(f'{m};{n};{m/n}')
ZeroDivisionError: division by zero So if you are perplexed as to what is happening to your async program, this decorator may help.

Reducing Debugging Woes with Python Async Decorator

I recently read a medium post where the author claimed that using is better than using While this might hold good in general cases, I think that generalisation was too broad and doesn’t do justice to either or For somebody who is new to Javascript making sense of these and decide which one to use can be a challenge. In this post I will list down things that I have learnt about these and how I decide when to use what. If you find anything interesting/useful in this article you can highlight them using our chrome extension async-await promises. async-await promises. https://bit.ly/highlights-extension I read somewhere that is syntactical sugar for using . So before getting to know or deciding on which approach to you use make sure that you have a better understanding of promises. I captured my learnings in the post . If you do not have time to go through the full article checkout the thumb rules that I follow. async-await promises async-await Understanding promises in Javascript Thumb Rules for using Promises Use promises whenever you are using asynchronous or blocking code. maps to and maps to for all practical purposes. resolve then reject catch Make sure to write both and methods for all the promises. .catch .then If something needs to be done in both the cases use .finally We only get one shot at mutating each promise. We can add multiple handlers to a single promise. The return type of all the methods in object whether they are static methods or prototype methods is again a Promise Promise In the order of the promises are maintained in values variable irrespective of which promise was first resolved. Promise.all Once you have wrapped your head around promises checkout It helps you to write code that is much more readable. When not used properly it has its downsides. You can read to checkout my learnings. If you do not have time to read the full article checkout my thumb rules. async await. Understanding async-await in Javascript Thumb Rules for async-await Here are a list of thumb rules I use to keep my head sane around using and async await functions returns a promise. async functions use an implicit Promise to return its result. Even if you don’t return a promise explicitly function makes sure that your code is passed through a promise. async async blocks the code execution within the function, of which it( ) is a part. await async await statement There can be multiple statements within a single function. await async When using make sure to use for error handling. async await try catch Be extra careful when using within loops and iterators. You might fall into the trap of writing sequentially executing code when it could have been easily done in parallel. await is always for a single promise. await Promise creation starts the execution of asynchronous functionality. only blocks the code execution within the function. It only makes sure that next line is executed when the resolves. So if an asynchronous activity has already started then will not have an effect on it. await async promise await So should I use promises or async-await The answer is that we will use both. Following are the thumb rules I use to decide when to use and when to use promises async await returns a The converse is also true. Every function that returns a can be considered as async function promise. promise async function is used for calling an and wait for it to or . await async function resolve reject blocks the execution of the code within the function in which it is located. await async If the output of is dependent on output of then I use . function2 function1 await If two functions can be run in parallel create two different and then run them in parallel. async functions To run promises in parallel create an array of promises and then use Promise.all(promisesArray) Every time you use remember that you are writing blocking code. Over the time we tend to neglect this. await Instead of creating huge with many in it, it is better to create smaller This way we will be aware of not writing too much of blocking code. async functions await asyncFunction() async functions. Another advantage of using smaller is that you force yourself to think what are the async functions that can be run in parallel. async functions If your code contains blocking code it is better to make it an function. By doing this you are making sure that somebody else can use your function asynchronously. async By making async functions out of blocking code, you are enabling the user who will call your function to decide on the level of asynhronicity he wants. Hope this helps you decide easily when to use promises and when to use and when to use promises async-await If you liked this article and would like to read similar articles, don’t forget to clap. Click and drag to clap more than once. 50 is the limit. Understanding Series This article is a part of my learning series, which I call . You can read more above it Understanding X in this link. Understanding Javascript _I have had a kind of “love and hate” relationship with JavaScript. But nevertheless JavaScript was always intriguing…_hackernoon.com Understanding promises in JavaScript _Async and Await are extensions of promises. So if you are not clear about the basics of promises please get comfortable…_hackernoon.com Understanding async-await in Javascript _I recently read a medium post where the author claimed that using async-await is better than using promises. While this…_hackernoon.com Should I use Promises or Async-Await _I think npm was one of the reasons for quick adoption of nodejs. As of writing this article there are close 7,00,000…_hackernoon.com Understanding npm in Nodejs Understanding Linux _Survival guide for those who are starting with Linux_hackernoon.com Linux Survival Guide for Beginners _Monitor all your server infrastructure_hackernoon.com Understanding Zabbix _Ubuntu Firewall for humans_hackernoon.com Understanding UFW _Monitoring services and starting them on boot_hackernoon.com Linux Systemctl Understanding React _React is all about keeping your front-end code modular and reusable. Components play a crucial role in making React…_hackernoon.com Understanding Functional Components Understanding Learning _The dictionary meaning of a rabbit hole is as follows._medium.com The Thing about Rabbit Holes Understanding Cryptocurrencies _Price is an important indicator. But it can also be misleading in many cases._blog.goodaudience.com Why comparing cryptocurrency prices is wrong _It has been more than an year since I started investing in cryptocurrencies. Following Warren Buffet’s advice of “Never…_hackernoon.com Beginner’s Guide to “Investing in Cryptocurrencies”

Should I use Promises or Async-Await

Browsers now have native support for doing asyncronous calls via / . This is nice. It is essentially syntax support for promises. async await Unfortunately, the error handling story for all this not so nice. Real footage of the async error delegation mechanism To summarize, here’s what we’ve got to deal with: 2 non-equivalent ways throwing errors. 2 mostly equivalent ways of catching local errors. From 0 to 1 ways of catching promise errors globally. 1 way to throw errors that don’t get caught by any of these mechanisms. Ugh. How did we end up with this? Well, when adding new features to a system, if every feature number n has to interact with all of the existing n-1 features, you get an O(n²) growth in feature-feature interactions. So for a linear growth in features, you get a quadratic growth in complexity. This actually explains why most big software projects fail, and why disentangling features is so important. It’s also what has happened to async error handling. We started with simple callback functions. Then it turned out that callbacks was a mess, so we fixed all that with Promises. Then it turned out that Promises was a mess, so we fixed all that with async/await. So let’s dig into the current mess. Handling errors in promises locally Promises, they break before they’re madeSometimes, sometimes - The Strokes, in a post to the WHATWG mailing list Thrown errors When an error is thrown in an async function, you can catch it with a . So this works as you'd expect: try {} catch {} () {throws Error();} async function fails () { { fails();} (e) {console.log("that failed", e);}} async function myFunc try await catch This is syntax sugar for what you might have been doing with promises earlier: fails().catch(e => {console.log("That also failed", e);}); In fact, anywhere you use the keyword , you can remove and do the traditional and calls. This is because the keyword implicitly creates a for its function. await await .then() .catch() async Promise The only difference between these two is that the callback for has its own execution context, i.e. variable scope works like you'd expect it to. catch() Rejected promises With Promises, it turns out you have another way of throwing errors, other than using throw. You can call : reject() () { Promise((resolve, reject) => {reject( Error());});} function fails2 return new new () { { fails2();} (e) {console.log("that failed", e);}} async function myFunc2 try await catch Errors passed to can be caught with both and with the method. So you have two ways to throw errors, and two ways to catch errors. This is more complex than we'd like, but at least both way of catching errors will catch either way of throwing them, so the complexity here isn't fully as bad as it could have been. reject() try {} catch {} .catch() Errors thrown in a different call stack There’s more trouble to be had though. If you’re creating yourself, chances are you're using either a or a , or in some way calling a callback function when some operation is done. These callbacks will be called from a different call stack, which means that thrown errors will propagate to somewhere that is not your code. Promise setTimeout() setInterval() Consider this example: () { Promise((resolve, reject) => {setTimeout( () { Error();}, 100);});} function fails3 return new function throw new () { { fails3();} (e) {console.log("that failed", e); //<-- never gets called}} async function myFunc3 try await catch The error produced here is never caught by the , because it is thrown on a different call stack. Using the method would have the same problem. try {} catch {} .catch(() => {}) The way to have an error propagate across such callbacks is to use the reject() function, like so: () { Promise((resolve, reject) => {setTimeout( () {reject( Error());}, 100);});} function fails4 return new function new () { { fails4();} (e) {console.log("that failed", e); //<-- this gets called}} async function myFunc4 try await catch This is presumably the main reason why the reject/resolve paradigm was introduced in the first place. Sidenote: Why reject/resolve kind of sucks. Calling in a promise is much like doing , except for a major difference: It's just a function call, so it doesn't break the execution flow like does. This means you can write paradoxical code that both s and s, like this: reject(new Error()) throw Error() throw reject resolve () { Promise((resolve, reject) => {reject( Error());resolve("great success");});} function schrödinger return new new Here both and will be called. So which will win? The answer is whichever function was called . reject() resolve() first Now look at this weirdo: () { Promise((resolve, reject) => { resolve("huh"); //<-- this throw is executed});} () { { schrödinger2();} (e) {console.log("caught error", e);//<-- yet, this isn't reached}} function schrödinger2 return new throw async function callAsync try await catch Here the promise has a single line of code, a throw statement. Yet, the is never triggered. This is because resolve was called, and the rule still is that whatever was called is what wins. So the throw is executed, but it is silently swallowed by the runtime. This is bound to cause endless confusion. try {} catch {} first These problems happen because and are near duplicates of and . I'll claim that the only reason we have reject/resolve is to be able to move errors across call stack boundaries. But it's a mediocre fix for that, for several reasons. It only moves the errors you expect, so e.g. an unexpected NullReferenceException will not be moved across boundaries unless you explicitly call with it yourself. Also, the fact that it duplicates core language features causes problems, as seen above. resolve() reject() return throw reject() There’s a cleaner design for this. C# has had / since before people started talking about it in . There, exceptions thrown in the async callbacks are caught, and then rethrown such that they propagate to the site that is awaiting the async operation. JavaScript could implement this by providing substitutes for setTimeout and setInterval with new semantics for errors, and we could ditch this / stuff in favor of / . This would also cut down the Promises spec by 90%. async await JavaScript resolve reject return throw Handling errors in promises globally So we know how to catch errors with and similar mechanisms. What about when you want to set up a global catch-all handler for all unhandled errors, for example to log these errors to a server? try {} catch {} Well, how do you even tell if an error in a promise is unhandled? When dealing with promises, you have no way of knowing if an error will be handled some time in the . The promise might call , and some code might come along 10 minutes later and call on that promise, in which case the error will be handled. For this reason, the global error handler in Promise libraries like Q and Bluebird has been named , which is a fitting name. In native Promises, this function is called , but they still can only tell if a rejection has been unhandled . future reject() .catch(() => {}) onPossiblyUnhandledRejection onunhandledrejection so far You can set up your global handler like this: window.onunhandledrejection = (evt) { /*Your code*/ } function or: window.addEventListener("unhandledrejection", (evt) { }) function Here is an object of type . is the promise that was rejected, and holds whatever object was passed to the function. evt PromiseRejectionEvent evt.promise evt.reason reject() This is all nice and dandy, except for this: No one except Chrome implement it (well, Chrome, and Chromium based browsers). It is coming to Firefox, and presumably to Safari and Edge as well. But not yet. To make matters worse, there is no good work around for these browsers, other than not using native Promises, and relying on a library like Q or Bluebird instead. Hopefully native support will arrive for these browsers soon. Tracking errors in promises How do you track JavaScript errors that happen on your site? For this you can use , which is a JavaScript error tracking service. It instruments the browser with a global error handler, which then logs any uncaught errors that occur. Deployment is simply done by dropping in a script file. CatchJS <script src="//cdn.catchjs.com/catch.js"></script> With this, uncaught errors get logged, along with various telemetry, which can include screenshots and click trails. Because of the ambiguity of whether or not a Promise rejection will be handled in the future, CatchJS does not attach it self to the onunhandledrejection handler. If you want this, you can set up such forwarding manually. window.onunhandledrejection = (evt) {console.error(evt.reason);} function CatchJS will instrument console.error, so these errors will be logged to your remote persistent log, as well as to the developers console. Mahatma Gandhi said that a breach of promise is a base surrender of truth. Then again, he knew nothing about async error handling. Originally published at catchjs.com .

Async error handling: N² ways to shoot yourself in the foot

sad puppy waiting Async and Await are extensions of promises. So if you are not clear about the basics of promises please get comfortable with promises before reading further. You can read my post on Understanding Promises in Javascript. If you would like to highlights some parts of this article you can use our extension . You can search for your previous highlights from the dashboard on http://bit.ly/highlights-extension https://alpha.app.learningpaths.io/#/highlights I am sure that many of you would be using async and await already. But I think it deserves a little more attention. Here is a small test : If you can’t spot the problem with the below code then read on. for (name of ["nkgokul", "BrendanEich", "gaearon"]) {userDetails = await fetch(" " + name);userDetailsJSON = await userDetails.json();console.log("userDetailsJSON", userDetailsJSON);} https://api.github.com/users/ We will revisit the above code block later, once we have gone through async await basics. Like always is your friend. Especially checkout the definitions. Mozilla docs async and await From MDN An asynchronous function is a function which operates asynchronously via the event loop, using an implicit to return its result. But the syntax and structure of your code using async functions is much more like using standard synchronous functions. [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise "The Promise object represents the eventual completion (or failure) of an asynchronous operation, and its resulting value.") I wonder who writes these descriptions. They are so concise and well articulated. To break it down. The function operates asynchronously via event loop. It uses an implicit Promise to return the result. The syntax and structure of the code is similar to writing synchronous functions. And MDN goes on to say An function can contain an expression that pauses the execution of the async function and waits for the passed 's resolution, and then resumes the function's execution and returns the resolved value. Remember, the keyword is only valid inside functions. async [await](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/await "The await operator is used to wait for a Promise. It can only be used inside an async function.") Promise async await async Let us jump into code to understand this better. We will reuse the three function we used for understanding promises here as well. A function that returns a promise which resolves or rejects after n number of seconds. var promiseTRRARNOSG = ( = function() {return new Promise(function(resolve, reject) {let randomNumberOfSeconds = getRandomNumber(2, 10);setTimeout(function() {let randomiseResolving = getRandomNumber(1, 10);if (randomiseResolving > 5) {resolve({randomNumberOfSeconds: randomNumberOfSeconds,randomiseResolving: randomiseResolving});} else {reject({randomNumberOfSeconds: randomNumberOfSeconds,randomiseResolving: randomiseResolving});}}, randomNumberOfSeconds * 1000);});}); promiseThatResolvesRandomlyAfterRandomNumnberOfSecondsGenerator Two more deterministic functions one which resolve after n seconds and another which rejects after n seconds. var promiseTRSANSG = (promiseThatResolvesAfterNSecondsGenerator = function(n = 0) {return new Promise(function(resolve, reject) {setTimeout(function() {resolve({resolvedAfterNSeconds: n});}, n * 1000);});});var promiseTRJANSG = (promiseThatRejectsAfterNSecondsGenerator = function(n = 0) {return new Promise(function(resolve, reject) {setTimeout(function() {reject({rejectedAfterNSeconds: n});}, n * 1000);});}); Since all these three functions are returning promises we can also call these functions as asynchronous functions. See we wrote asyn functions even before knowing about them. If we had to use the function using standard format of promises we would have written something like this. promiseTRSANSG var promise1 = promiseTRSANSG(3);promise1.then(function(result) {console.log(result);}); promise1.catch(function(reason) {console.log(reason);}); There is a lot of unnecessary code here like anonymous function just for assigning the handlers. What async await does is it improves the syntax for this which would make seem more like synchronous code. If we had to the same as above in async await format it would be like result = await promiseTRSANSG(3);console.log(result); Well that look much more readable than the standard promise syntax. When we used the execution of the code was blocked. That is the reason that you had the value of the promise resolution in the variable . As you can make out from the above code sample, instead of the part the result is assigned to the variable directly when you use You can also make out that the part is not present here. That is because that is handled using error handling. So instead of using let us use Since this function can either resolve or reject we need to handle both the scenarios. In the above code we just wrote two lines to give you an easy comparison between the standard format and format. The example in next section gives you a better idea of the format and structure. await result .then await .catch try catch promiseTRSANS promiseTRRARNOSG async await General syntax of using async await async function testAsync() {for (var i = 0; i < 5; i++) {try {result1 = await promiseTRRARNOSG();console.log("Result 1 ", result1);result2 = await promiseTRRARNOSG();console.log("Result 2 ", result2);} catch (e) {console.log("Error", e);} finally {console.log("This is done");}}} testAsync(); From the above code example you can make out that instead of using the promise specific error handling we are using the more generic approach of using for error handling. So that is one thing less for us to remember and it also improves the overall readability even after considering the try catch block around our code. So based on the level of error handling you need you can add any number of blocks and make the error messages more specific and meaningful. try catch catch Pitfalls of using async and await makes it much more easier to use promises. Developers from synchronous programming background will feel at home while using and . This should also alert us, as this also means that we are moving towards a more synchronous approach if we don’t keep a watch. async await async await The whole point of javascript/nodejs is to think asynchronous by default and not an after though. generally means you are doing things in sequential way. So make a conscious decision whenever you want to use to async await. async await Now let us start analysing the code that I flashed at your face in the beginning. for (name of ["nkgokul", "BrendanEich", "gaearon"]) {userDetails = await fetch(" " + name);userDetailsJSON = await userDetails.json();console.log("userDetailsJSON", userDetailsJSON);} https://api.github.com/users/ This seems like a harmless piece of code that fetches the github details of three users Right. That is true. That is what this function does. But it also has some unintended consequences. “nkgokul”, “BrendanEich”, “gaearon” Before diving further into the code let us build a simple timer. startTime = performance.now();  //Run at the beginning of the code function executingAt() {return (performance.now() - startTime) / 1000;} Now we can use wherever we want to print the number of seconds that have surpassed since the beginning. executingAt async function fetchUserDetailsWithStats() {i = 0;for (name of ["nkgokul", "BrendanEich", "gaearon"]) {i++;console.log("Starting API call " + i + " at " + executingAt());userDetails = await fetch(" " + name);userDetailsJSON = await userDetails.json();console.log("Finished API call " + i + "at " + executingAt());console.log("userDetailsJSON", userDetailsJSON);}} https://api.github.com/users/ Checkout the output of the same. async-await analysed As you can find from the output, each of the await function is called after the previous function was completed. We are trying to fetch the details of three different users It is pretty obvious that output of one API call is in noway dependent on the output of the others. “nkgokul”, “BrendanEich”, “gaearon” The only dependence we have is these two lines of code. userDetails = await fetch(" " + name);userDetailsJSON = await userDetails.json(); https://api.github.com/users/ We can create the object only after getting the . Hence it makes sense to use await here that is within the scope of getting the details of a single user. So let us make an for getting the details of the single user. userDetailsJSON userDetails async async function fetchSingleUsersDetailsWithStats(name) {console.log("Starting API call for " + name + " at " + executingAt());userDetails = await fetch(" " + name);userDetailsJSON = await userDetails.json();console.log("Finished API call for " + name + " at " + executingAt());return userDetailsJSON;} https://api.github.com/users/ Now that the is async we can use this function to fetch the details of the different users in parallel. fetchSingleUsersDetailsWithStats async function fetchAllUsersDetailsParallelyWithStats() {let singleUsersDetailsPromises = [];for (name of ["nkgokul", "BrendanEich", "gaearon"]) {let promise = fetchSingleUsersDetailsWithStats(name);console.log("Created Promise for API call of " + name + " at " + executingAt());singleUsersDetailsPromises.push(promise);}console.log("Finished adding all promises at " + executingAt());let allUsersDetails = await Promise.all(singleUsersDetailsPromises);console.log("Got the results for all promises at " + executingAt());console.log(allUsersDetails);} When you want to run things in parallel, the thumb rule that I follow is Create a promise for each async call. Add all the promises to an array. Then pass the promises array to Promise.all This in turn returns a single promise for which we can use await When we put all of this together we get startTime = performance.now();async function fetchAllUsersDetailsParallelyWithStats() {let singleUsersDetailsPromises = [];for (name of ["nkgokul", "BrendanEich", "gaearon"]) {let promise = fetchSingleUsersDetailsWithStats(name);console.log("Created Promise for API call of " + name + " at " + executingAt());singleUsersDetailsPromises.push(promise);}console.log("Finished adding all promises at " + executingAt());let allUsersDetails = await Promise.all(singleUsersDetailsPromises);console.log("Got the results for all promises at " + executingAt());console.log(allUsersDetails);} async function fetchSingleUsersDetailsWithStats(name) {console.log("Starting API call for " + name + " at " + executingAt());userDetails = await fetch(" " + name);userDetailsJSON = await userDetails.json();console.log("Finished API call for " + name + " at " + executingAt());return userDetailsJSON;}fetchAllUsersDetailsParallelyWithStats(); https://api.github.com/users/ The output for this is Promises run in parallel with timestamps As you can make out from the output, promise creations are almost instantaneous whereas API calls take some time. We need to stress this as time taken for promises creation and processing is trivial when compared to IO operations. So while choosing a promise library it makes more sense to choose a library that is feature rich and has better dev experience. Since we are using all the API calls were run in parallel. Each API is taking almost 0.88 seconds. But since they are called in parallel we were able to get the results of all API calls in 0.89 seconds. Promise.all In most of the scenarios understanding this much should serve us well. You can skip to Thumb Rules section. But if you want to dig deeper read on. Digging deeper into await For this let us pretty much limit ourselves to function. The outcome of this function is more deterministic and will help us identify the differences. promiseTRSANSG Sequential Execution startTime = performance.now();var sequential = async function() {console.log(executingAt());const resolveAfter3seconds = await promiseTRSANSG(3);console.log("resolveAfter3seconds", resolveAfter3seconds);console.log(executingAt());const resolveAfter4seconds = await promiseTRSANSG(4);console.log("resolveAfter4seconds", resolveAfter4seconds);end = executingAt();console.log(end);}sequential(); Sequential Execution Parallel Execution using Promise.all var parallel = async function() {startTime = performance.now();promisesArray = [];console.log(executingAt());promisesArray.push(promiseTRSANSG(3));promisesArray.push(promiseTRSANSG(4));result = await Promise.all(promisesArray);console.log(result);console.log(executingAt());}parallel(); Parallel execution using promises Concurrent Start of Execution asynchronous execution starts as soon as the is created. just blocks the code within the function until the promise is resolved. Let us create a function which will help us clearly understand this. promise await async var concurrent = async function() {startTime = performance.now();const resolveAfter3seconds = promiseTRSANSG(3);console.log("Promise for resolveAfter3seconds created at ", executingAt());const resolveAfter4seconds = promiseTRSANSG(4);console.log("Promise for resolveAfter4seconds created at ", executingAt()); resolveAfter3seconds.then(function(){console.log("resolveAfter3seconds resolved at ", executingAt());}); resolveAfter4seconds.then(function(){console.log("resolveAfter4seconds resolved at ", executingAt());}); console.log(await resolveAfter4seconds);console.log("await resolveAfter4seconds executed at ", executingAt());console.log(await resolveAfter3seconds);console.log("await resolveAfter3seconds executed at ", executingAt()); };concurrent(); Concurrent start and then await From previous post we know that is even driven. That is is executed as soon as the promise is resolved. So let us use and to identify when our promises are actually resolved. From the output we can see that is resolved after 3 seconds and is executed after 4 seconds. This is as expected. .then .then resolveAfter3seconds.then resolveAfter4seconds.then resolveAfter3seconds resolveAfter4seconds Now to check how affects the execution of code we have used await console.log(await resolveAfter4seconds);console.log(await resolveAfter3seconds); As we have seen from the output of resolved one second before . But we have the for and then followed by for .then resolveAfter3seconds resolveAfter4seconds await resolveAfter4seconds await resolveAfter3seconds From the output we can see that though was already resolved it got printed only after the output of was printed. Which reiterates what we had said earlier. only blocks the execution of next lines of code in function and doesn’t affect the promise execution. resolveAfter3seconds console.log(await resolveAfter4seconds); await async Disclaimer mentions that is still serial and using is truly parallel. I have not been able to understand the difference and would love to hear back if anybody has figured out their heard around the difference. MDN documentation Promise.all .then Thumb Rules Here are a list of thumb rules I use to keep my head sane around using and async await functions returns a promise. async functions use an implicit Promise to return its result. Even if you don’t return a promise explicitly function makes sure that your code is passed through a promise. async async blocks the code execution within the function, of which it( ) is a part. await async await statement There can be multiple statements within a single function. await async When using make sure to use for error handling. async await try catch If your code contains blocking code it is better to make it an function. By doing this you are making sure that somebody else can use your function asynchronously. async By making async functions out of blocking code, you are enabling the user who will call your function to decide on the level of asynhronicity he wants. Be extra careful when using within loops and iterators. You might fall into the trap of writing sequentially executing code when it could have been easily done in parallel. await is always for a single promise. If you want to multiple promises(Run this promises in parallel) create an array of promises and then pass it to the function. await await Promise.all Promise creation starts the execution of asynchronous functionality. only blocks the code execution within the function. It only makes sure that next line is executed when the resolves. So if an asynchronous activity has already started then will not have an effect on it. await async promise await Please point out if I am missing something here or if something can be improved. If you didn’t have anything new to learn from this post and you are from Bangalore then we would like to hire you. We have a opening in our product team. Leave a private message if you would like to know further. If you liked this article and would like to read similar articles, don’t forget to clap. Click and drag to clap more than once. 50 is the limit. Understanding Series This article is a part of my learning series, which I call . You can read more above it Understanding X in this link. Understanding Javascript _I have had a kind of “love and hate” relationship with JavaScript. But nevertheless JavaScript was always intriguing…_hackernoon.com Understanding promises in JavaScript _Async and Await are extensions of promises. So if you are not clear about the basics of promises please get comfortable…_hackernoon.com Understanding async-await in Javascript _I recently read a medium post where the author claimed that using async-await is better than using promises. While this…_hackernoon.com Should I use Promises or Async-Await _I think npm was one of the reasons for quick adoption of nodejs. As of writing this article there are close 7,00,000…_hackernoon.com Understanding npm in Nodejs Understanding Linux _Survival guide for those who are starting with Linux_hackernoon.com Linux Survival Guide for Beginners _Monitor all your server infrastructure_hackernoon.com Understanding Zabbix _Ubuntu Firewall for humans_hackernoon.com Understanding UFW _Monitoring services and starting them on boot_hackernoon.com Linux Systemctl Understanding React _React is all about keeping your front-end code modular and reusable. Components play a crucial role in making React…_hackernoon.com Understanding Functional Components Understanding Learning _The dictionary meaning of a rabbit hole is as follows._medium.com The Thing about Rabbit Holes Understanding Cryptocurrencies _Price is an important indicator. But it can also be misleading in many cases._blog.goodaudience.com Why comparing cryptocurrency prices is wrong _It has been more than an year since I started investing in cryptocurrencies. Following Warren Buffet’s advice of “Never…_hackernoon.com Beginner’s Guide to “Investing in Cryptocurrencies”

Understanding async-await in Javascript

One of the most recent additions to ECMAScript/JavaScript are async generators; these build on top of regular generators to add async functionality. In case you’re not already familiar with them, generators are special functions which can ‘return’ multiple times using a statement like this: yield // Generator functions are declared with an asterisk (function*) function* (start, end) {let n = 0;for (let i = start; i < end; i++) {n += 1;yield i;}return n;} makeRangeGenerator let rangeGenerator = (0, 3); makeRangeGenerator console.log(rangeGenerator.next()); // { value: 0, done: false }console.log(rangeGenerator.next()); // { value: 1, done: false }console.log(rangeGenerator.next()); // { value: 2, done: false }console.log(rangeGenerator.next()); // { value: 3, done: true } Generator functions are different from regular functions in at least two fundamental ways: A generator function can yield (~return) values multiple times without exiting and losing its internal state. A caller can push values back into the generator function at the point where it last yielded — the generator will resume execution from that point until it encounters the next yield. Note that you can iterate over the output of a generator using a loop like this: for-of for (let integer of rangeGenerator) {console.log(integer);} ^ This is slightly cleaner than using in a regular loop; in this case our variable will be the raw value instead of the object . rangeGenerator.next() for integer {value: ..., done: false} Sometimes, we may want to pass values into a generator like this: function* pushPullGenerator() {let n = 0;while (true) {n = yield n * 2;}} let gen = pushPullGenerator(); console.log(gen.next());  // { value: 0, done: false }console.log(gen.next(2)); // { value: 4, done: false }console.log(gen.next(3)); // { value: 6, done: false }console.log(gen.next(4)); // { value: 8, done: false } ^ Here we’re passing values to the generator using calls. Note that the first invocation of doesn’t have any arguments; this is because the first in the generator is different from the others; it behaves more like a plain statement. Subsequent calls to will both the specified value into the generator function (in place of the last ) and then a new value out of it (on the following ). gen.next(...) gen.next() yield return gen.next(value) pull yield push yield Coming up with use cases for generators might not be obvious at first. If overused, generators could make code more difficult to follow. However, if used correctly, generators can offer some unique capabilities. In this guide, we will look at a specific kind of generator; the async generator. Async generators (as in ) are like regular generators except that instead of yielding normal values, they yield objects which resolve asynchronously. This allows us to do some useful things; in particular, async generators are ideal for representing asynchronous streams of data. async/await Promise If you’ve used before, you should already be familiar with the idea of consuming asynchronous streams of data in a declarative (reactive) way. Async generators can be used as an alternative to Observables; one of the main benefits of using async generators over Observables is that they will make your code look more like regular synchronous JavaScript; this should make it more succinct and readable. RxJS Observables Here is an example of an async generator which streams some integers: function waitForDelay(delay) {return new Promise((resolve) => {setTimeout(() => {resolve();}, delay);});} async function* createRandomStream(randomness) {let n = 0;while (true) {let randomDelay = Math.round(Math.random() * randomness);await waitForDelay(randomDelay);yield n++;}} async function startConsumingStream() {let randomStream = createRandomStream(1000);for await (let value of randomStream) {console.log(value);}} startConsumingStream(); ^ The in this case is a stream of integers which increments 0, 1, 2, 3, 4, … — it’s only random in the sense that the delay period between each iteration of the generator will be a random value between 0 and 1000 milliseconds; the result is that you can see the logging of numbers accelerate and decelerate randomly. Note that, in this case, we’ve used a loop to iterate over the async generator; this is the async equivalent of the loop which we used earlier to iterate over a normal generator. As a challenge; you could try to modify the code above to use a trigonometric function like to generate the delay value; it should only require a one-line change. randomStream for-await-of for-of Math.sin(...) Random streams are fun, but what about something more useful? For example; maybe we want each iteration of our generator to correspond to an event; like receiving a message from a user. We can do something like this: // If the receiveMessage function is called before we// started consuming the data stream, then we don't care about that// data. In this case the function will be a no-op. let receiveMessage = function () {}; function waitForNextMessage() {return new Promise((resolve) => {receiveMessage = resolve;});} async function* createMessageStream() {while (true) {yield waitForNextMessage();}} async function startConsumingMessageStream() {let messageStream = createMessageStream();for await (let message of messageStream) {console.log(message);}} startConsumingMessageStream(); setTimeout(() => {receiveMessage('Hello');}, 500); setTimeout(() => {receiveMessage('world');}, 1000); setTimeout(() => {receiveMessage('!!!');}, 3000); ^ The trick here is to assign the function from the in the function to our own function; that way it can be invoked anytime from anywhere else in our code — the message in this case is triggered by a but it could also have come from any other source such as an event from an HTTP request or a WebSocket message. Note that because we’re using async/await, it doesn’t cost any CPU to wait indefinitely. resolve Promise waitForNextMessage receiveMessage setTimeout A small problem… There is a potential problem with the approach above; what would happen if you tried calling synchronously multiple times? Like this: receiveMessage(message) setTimeout(() => {// Call receiveMessage multiple times synchronously.receiveMessage('Hello');receiveMessage('A');receiveMessage('B');receiveMessage('C');}, 500); setTimeout(() => {receiveMessage('world');}, 1000); setTimeout(() => {receiveMessage('!!!');}, 3000); ^ In this case, we would miss the messages ‘A’, ‘B’ and ‘C’ — this is because the returned by our function will resolve asynchronously on the next tick. So if we call multiple times while in the same call stack, it’s going to end up trying to resolve the same multiple times (instead of different promises for each message). There are several ways to get around this issue. One approach is to modify so that it buffers messages synchronously and resolves the promise with the buffer; that way we can keep pushing more messages into the buffer synchronously before the finishes resolving. Promise waitForNextMessage receiveMessage Promise receiveMessage Promise The solution might look like this: let receiveMessage = function () {}; function waitForNextMessageList() {let messageBuffer = [];return new Promise((resolve) => {receiveMessage = (message) => {messageBuffer.push(message);// Only resolve the promise for the first message in the// current call stack. After that we can keep pushing messages// to the buffer.if (messageBuffer.length === 1) {resolve(messageBuffer);}};});} async function* createMessageListStream() {while (true) {yield waitForNextMessageList();}} async function* createMessageStream() {let messageListStream = createMessageListStream();for await (let messageList of messageListStream) {for (let message of messageList) {yield message;}}} async function startConsumingMessageStream() {let messageStream = createMessageStream();for await (let messages of messageStream) {console.log(messages);}} startConsumingMessageStream(); setTimeout(() => {receiveMessage('Hello');receiveMessage('A');receiveMessage('B');receiveMessage('C');}, 500); setTimeout(() => {receiveMessage('world');}, 1000); setTimeout(() => {receiveMessage('!!!');}, 3000); This code became slightly more complex after we added the buffering, but if you think about it from the stream consumer’s point of view, it still looks pretty simple: async function startConsumingMessageStream() {let messageStream = createMessageStream();for await (let message of messageStream) {console.log(message);}} startConsumingMessageStream(); A usable solution The code above may be useful in some situations where a stream only has a single consumer. If we call multiple times, the consumers will compete for each message and only one of them will work. This is because our stream currently behaves like a FIFO queue so each new item pushed into it can only have one consumer. startConsumingMessageStream() For practical scenarios, we may want to allow streams to be shared between multiple consumers. Also, we should expose this functionality through a simple object. Iterable After trying out several different approaches, I settled with a singly-linked-list-based solution. Check out WritableAsyncIterableStream: https://github.com/SocketCluster/writable-async-iterable-stream If you would like to use async iterables as alternatives to EventEmitter objects, you may want to also check out StreamDemux: https://github.com/socketcluster/stream-demux

Using async generators as custom data streams

Simplify API and middleware tests with lesser-known features This is a transcript of a presentation given at . October Node.js Berlin Meetup What is Koa and what is Jest Let's briefly describe the libraries we will be working with. ( ) is a JavaScript web server framework. It was developed by the people behind a more famous Express as a lightweight and expressive ‘spiritual successor.’ Koa koajs.com It, too, is middleware based. However, it comes bundled with none. Yes, no router, no body parser, and no proxy. Utilising ES2017 functions brings user-friendly functions and end-to-end flow control. (Are you not familiar with ? Read by . We will use them a lot.) That means we can say ‘bye-bye’ to the callback hell, we know from Express. We will be using the second version; in Koa v1 generators provided the flow control. async/await async/await this helpful intro Mostafa Gaafar ( ) is a modern unit testing framework from Facebook. It brought several concepts to mainstream JavaScript testing: zero configuration, first-class mocking, and snapshots. Jest facebook.github.io/jest If we say Koa is a (spiritual) successor of Express, Jest can be described as a (spiritual) successor of Jasmine and Expect (which was later ). donated to Jest includes a better async/Promise support, which–I think–hasn't received the attention it deserves. I hope it is clear, that is what we will use later. The May release Personally, I have much more experience with Jest, which I use daily for testing React-based apps. Koa, on the other hand, is my hobby which I've only used on private projects. Testing a middleware A Koa middleware has–in general–this form: const greetings = async (ctx, next) => {ctx.body = 'Hello.' await next() ctx.body += ' Remember to subscribe.'} const app = new Koa()app.use(greetings)app.listen(3000) It receives context which includes information about the request and response. Mutating this object is the only way the middleware communicates. There is no return value; if there were it would be ignored. Testing the middleware, therefore, means observing changes on the context. ctx The second argument is the callback that suspends the current middleware and passes control to the next one. This callback must either be awaited or not be invoked at all–turning the middleware order execution. Of course, it may not be called more than once. Read more details in the well-written , from where I borrowed this animation guide Simple test A simple test of the middleware above can look like this: greeting test('greetings works', async () => {const ctx = {} await greetings(ctx, () => {}) expect(ctx.body).toBe('Hello. Remember to subscribe.')}) First, yes you may use in Jest. The framework will wait for all asynchronous operations to finish. async The callback is an empty function–that is the required minimum. It just returns the flow immediately back to our function. next The context object is a mock. We could provide other data like requested URL or headers. When the whole middleware finishes, we run assertions on the context object. As you can see, we cannot distinguish when the changes happened–‘before’ or ‘after’ , we only know the result. However, for many middleware this is enough. Especially those that run something small only before the next middleware, or only after. await next() What if we need more? Before-and-after test Reading from a file, logging time, and generating ETag are some of the examples when it's essential if it's run before or after passing to next middleware. Such test could look like the following: test('greetings works before-and-after', async () => {const ctx = {} const next = jest.fn(() => {expect(ctx.body).toBe('Hello.')  // (1)ctx.body += ' I am content.'}) await greetings(ctx, next) expect(next).toHaveBeenCalledTimes(1) expect(ctx.body).toBe(             // (2)'Hello. I am content. Remember to subscribe.')}) Whoa, that has grown a bit. But fear not, it's easy. The most important change is inside the callback. It's not a noop anymore. This is the place, marked with (1), where we test how changed ‘before.’ Here, we can also prepare for the ‘after’ part, which is asserted at (2). next ctx Please note, we wrap in so we can check it run and the assertions in it passed. Otherwise, they could be skipped by not calling . You can remove it if you test it in a separate test. Alternatively, we could add on top –and keep it updated. next jest.fn await next() expect.assertions(2) A small warning: I've seen attempts to divide the test for the two parts by calling the middleware without . That is wrong and dangerous as potentially the ‘after’ part may be called, too. Moreover, it would not work if there were asynchronous operations in the ‘before.’ await Complete middleware test The following is an updated example that uses snapshot testing. The added benefit is that it will catch additional changes, you might otherwise miss. It also shows how to test a function call on a utility provided by Koa, in our case. It uses a short version how to test the number of calls and arguments in each call, order sensitive. response.set test('greetings works complete', async () => {const ctx = {response: { set: jest.fn() }/* ADD OTHER MOCKS */} const next = jest.fn(() => {expect(ctx).toMatchSnapshot()}) await expect(greetings(ctx, next)).resolves.toBeUndefined() expect(next).toHaveBeenCalledTimes(1) expect(ctx).toMatchSnapshot()expect(ctx.response.set.mock.calls).toMatchSnapshot()}) The last enhancement, you can see there, is matcher added in Jest 20. It does two things: checks the middleware does not return anything and provides better messages in case something throws. Compare: .resolves Before: Read error After: Expected received Promise to resolve, instead it rejected to value[Error: Read error] The difference would be even more pronounced when we would expect an error. For more, watch my presentation with examples: . Async testing in Jest What next? The small units are tested. Does that mean we are done? No. I like this GIF: Still love this one. Unit testers be like: “Looks like it’s working” — Kent C. Dodds We need confidence. Confidence our app works for the end user. We build it up when we pretend to use the app. Testing full API The whole app is technically one middleware which is a of all applied middleware, written by us or taken from libraries. It could be tempting to obtain the list with and test it the same way as described above. That would be wrong. composition app.middleware On top of composing middleware, Koa wraps the native response and request objects and does few other things. We want to test the whole app. It's not a unit test, let's call this an API test. Jest cannot do this by itself, for HTTP assertions we will use . Supertest is a small wrapper over . The great benefit is that it supports promises from the box. supertest SuperAgent Sample app and test boilerplate The sample app has been taken (with some modifications) from 's article . Read it! It goes a bit slower and includes a step-by-step guide to get it all running. Valentino Gagliardi A clear and concise introduction to testing Koa with Jest and Supertest Here I will show few other ways to test the app. // server/index.jsconst Koa = require('koa')const Router = require('koa-router')const router = new Router() const app = new Koa() router.get('/', async ctx => {ctx.body = {data: 'Sending some JSON',person: {name: 'Ferdinand',lastname: 'Vaněk',role: 'Brewery worker',age: 42}}}) app.use(router.routes()) module.exports = app It is important we export the koa server instance before we call . This way, in the test, we have access to the , and we will avoid one of the most common mistakes in API testing: not closing the server and subsequent memory leaks. Supertest will open and the server for us. app.listen(3000) app.callback close A basic boilerplate of a test is pretty straightforward: // test/root.spec.js const request = require('supertest')const app = require('../server') test('root route', async () => {const response = await request(app.callback()).get('/') expect(response).toBeDefined() // @TODO}) Firstly, we describe the request: specify path, set headers, or append data. The result is a response promise. We await it to get the content. Secondly, we run assertions on the response. In the following we will replace placeholder-like in the boilerplate with something more sophisticated. expect(response).toBeDefined Item-level assertions Of course, is just an other JS object, so we can use the usual assertions. This is what Valentino does: response expect(response.status).toEqual(200)expect(response.type).toEqual('application/json')expect(response.body.data).toEqual('Sending some JSON')expect(Object.keys(response.body.person)).toEqual(expect.arrayContaining(['name', 'lastname', 'role', 'age'])) It's great, except the last expect. That one is confusing and doesn't read well. In terms of readability and maintainability, I prefer more expressive syntax which is part of Expect. Object equality expect(response.body).toEqual(expect.objectContaining({person: {name: expect.anything(),lastname: expect.any(String),role: expect.stringMatching(/^Brewery/),age: expect.any(Number)}})) Here, we describe the structure. Think of Flow or PropTypes. Expect has several utilities: , , , , and some . [expect.anything()](http://facebook.github.io/jest/docs/en/expect.html#expectanything) [expect.any(constructor)](http://facebook.github.io/jest/docs/en/expect.html#expectanyconstructor) [expect.stringContaining(string)](http://facebook.github.io/jest/docs/en/expect.html#expectstringcontainingstring) [expect.stringMatching(regexp)](http://facebook.github.io/jest/docs/en/expect.html#expectstringmatchingregexp) others Jest will try to match the whole object. So when we would add, say, field , it would fail. To ignore other properties use as we do with the whole body (remember it also includes field ). nationality expect.objectContaining data Snapshots There is an alternative: Snapshots. As might have noticed, they are opposite to TDD. One cannot reasonably follow TDD and use snapshots. They show their strength when we either gradually build the API up bit-by-bit in the watch mode, or when we add tests for already working servers. Starting with snapshots cannot get any simpler: expect(response.body).toMatchSnapshot() And the snapshot saved: // test/__snapshots__/root.spec.js.snap exports[`root route with snapshots 1`] = `Object {"data": "Sending some JSON","person": Object {"age": 42,"lastname": "Vaněk","name": "Ferdinand","role": "Brewery worker",},}`; I will repeat it: it cannot get any simpler. Yet, it makes sure we return what we need and nothing extra–which is equally important. Not only Koa Of course, everything we've shown applies to other frameworks as well. For example, API testing is the same for Express–how convenient for refactoring! All code is available on GitHub: . robinpokorny/jest-example-koa Please, if you have any comments or suggestions reach to me. I love to solve puzzles, so I'll look at any problem you send me about Jest. Slides: Slides from the presentation in Berlin Related: by A clear and concise introduction to testing Koa with Jest and Supertest Valentino Gagliardi by An Introduction to Building TDD RESTful APIs with Koa 2, Mocha and Chai Valentino Gagliardi by API testing with Jest Koen van Gilst (GitHub Issue) Testing async/await middleware? (recording of presentation) Async testing in Jest by Snapshot Testing APIs with Jest Dave Ceddia (recording of presentation) Snapshot testing in Jest 👏 . If you like this post, please don’t forget to give a below Every clap notification is a motivational boost for me. If you would like to learn more, I recently started a YouTube channel about JavaScript. I post new video every week, so consider subscribing. Be there from the beginning and help me get better. _JavaScript is my passion: I like to write JavaScript, I like to read JavaScript, and I like to talk JavaScript._www.youtube.com/c/robinpokorny Robin Pokorny on YouTube

Async testing Koa with Jest

is a really cool and useful library. If we are able to use the functions with , the JavaScript life will be more comfortable.I would like to write the same as function’s way as much as possible. Because if we already know how to use , we don’t need to learn new functions. The example code is written as below. Lo-Dash Async/Await Lo-Dash Lo-Dash // example 1const array = [1, 2, 3];const result = _.map(array, syncFunc); // ↓ async/await const result = await _.map(array, asyncFunc); function syncFunc(n) {return n * 2;}function asyncFunc(n) {return new Promise(resolve => setTimeout(resolve, 10, n * 2));} The makes our code clean, we would also like to use it. _.chain // example 2const array = [1, 2, 3];const result = _.chain(array).map(syncFunc).sum().value(); // ↓ async/await const result = await _.chain(array).map(asyncFunc).sum().value(); Convert Lo-Dash functions to asynchronous functions library which I have developed has the function. It converts functions to asynchronous functions and assigns them into . The usage is as below. Aigle mixin Lo-Dash Aigle const _ = require('lodash');const Aigle = require('aigle'); Aigle.mixin(_); // example 1const array = [1, 2, 3];const result = await Aigle.map(array, asyncFunc); // example 2const result = await Aigle.chain(array).map(asyncFunc).sum().value(); // or const result = await Aigle.resolve(array).map(asyncFunc).sum(); You will be able to use the functions asynchronously. Furthermore, already has many functions which have the same functionality as functions, these functions won’t be converted because they are already optimized. Lo-Dash Aigle Lo-Dash Conclusion will make your code clean the same as . If you are interested in , I would love you to read the following articles. Let’s enjoy using . Aigle Lo-Dash Aigle Async/Await Aigle Reference Aigle Lo-Dash How to make the fastest Promise library Aigle vs Bluebird

How to use Lo-Dash functions with Async/Await

In this article I will describe how the ES2017 functions are essentially a play between two older features: and , both of which were added earlier to the language in the ES2016 specification. async JavaScript generators promises Before you start reading .. This article is an introduction to , or functions. not promises generators async The goal of this article is to describe how functions can be realised using and . only async promises generators It does offer any opinion whether functions are better or worse than the other approach. not async The code examples used in this article are ingeniously contrived for easier explanation. They are meant for any serious use. not But why .. ? Since functions are now natively supported, what is the need to understand how they work? async Well, apart from the obvious reason of , an important reason is supporting older platforms. If you want your code using these newer features to run on older browser versions or older Node.js versions, you would be required to use like to transform these newer features into older features. curiosity tools Babel Therefore, a solid understanding of how functions get decomposed into and can come in handy when reading/debugging your transformed code. For example, here is a simple example of an function : async generators promises async This function gets transformed by Babel into the following ES2016 code (don’t worry about understanding it right now, we will cover it later) : Babel output for the previous async function (ES2016) They look really different! However, if you understand how functions actually work, then this transformation is fairly obvious. async Another fun fact, browsers also implement functions in a similar fashion i.e. they the code to use and quite similar to Babel. async transform async generators promises Okay, so how does it happen ? Sometimes in order to understand how something works, the best way is to build it yourself. So let’s flip the question: Imagine we are given a piece of code that uses functions, how can we rewrite it using only and functions? async promises generator Here’s an function : async It performs three asynchronous tasks, one after the other where each task depends on the completion of the previous task. Finally, it returns the result of the last task. How can we write it using generators ? Let’s quickly recap how they work. Here’s a simple generator function : Generators are functions which can be exited and later re-entered. This generator-function has some interesting aspects (lifted from the docs) : gen MDN When a generator function is called, its body is not executed right away. Instead it returns an iterator-object which adheres to the i.e. it has a method. iterator protocol next The only way to execute the body of is by calling the method on its iterator-object. Every time the method is called, its body is executed until the next expression. The value of this expression is returned from the iterator. gen next next [yield](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/yield) This method also accepts an argument. Calling it with an argument replaces the current expression with the argument and resumes the execution till the next expression. next yield yield To elucidate (very, very crudely) .. A generator-function gets executed (i.e. one yield-expression at a time), by its iterator (the method). yield-by-yield next Every has a → → behaviour, so to say. yield give halt take It out the value of the current yield-expression, to the iterator. gives It then at this point, until the iterator’s method is called again. halts next When the method is called again, it the argument from it and replaces the currently halted yield-expression with it. It then moves to the next . next takes yield You may want to read the above summary again or refer to the amazing MDN docs ! But how does this help us ? By now you would be wondering, how do the generator functions help our situation? We need to model an asynchronous flow where we have to wait for certain tasks to finish before proceeding ahead. But so far in our discussion everything has been synchronous. How can we do that? Well, the most important insight here is that the generator-functions can yield too. promises A function can a (for example an async task), and its iterator can be controlled to for this to resolve (or reject), and then proceed with the resolved (or rejected) value. This pattern of weaving a an iterator with yielded allows us to model our requirement like this : generator yield promise halt promise promises (Notice how this generator function resembles our _async_ function!) But this is only half the story. Now we need a way to execute its body. We need a function that can control the iterator of this function to every time a is yielded and proceeds once it resolves (or rejects). It sounds complicated, but is very simple to implement, as shown below : generator halt promise A function that executes a generator function. (Only for explanation, ) do not use it ! Now we can execute our function using this function as shown below: generator init runner Use `runner` to execute the body of `init`. And that’s it! This combination of a function and our function achieves a similar outcome as the original function. runner generator async Please note that this runner function is only for demonstrating the concept. It is not suitable for any serious use. If you are looking for a proper implementation, you can find it here . To summarise We started with an function and then we wrote an identical implementation using and . That is, the following two pieces of code will have a similar effect : async generators promises Further exercises In the beginning of this article, we looked at how Babel transforms code to ES2016 code using and . You can now revisit that transformed code and compare how our function is similar to the function. In fact, that function is the foolproof version of our extremely simple function. async generators promises runner _asyncToGenerator _asyncToGenerator runner If you are still interested, you can go another step forward i.e. transform functions to ES2015 code i.e. without . For this you would have to emulate themselves (using a stateful busy loop with cases for e.g. see the ). async generators generators switch regenerator project I hope this explanation clears up the mystery behind functions. They offer a simpler syntax and therefore less code noise. The proposal for functions states that : async async The introduction of Promises and Generators in ECMAScript presents an opportunity to dramatically improve the language-level model for writing asynchronous code in ECMAScript. Thanks to , & for providing their feedback to improve this article. Akos Alisa Kristian

Stories, News & Mentions about ASYNC

Async’s Stories on HackerNoon