You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

132 lines
3.3 KiB

4 years ago
  1. # Async-Limiter
  2. A module for limiting concurrent asynchronous actions in flight. Forked from [queue](https://github.com/jessetane/queue).
  3. [![npm](http://img.shields.io/npm/v/async-limiter.svg?style=flat-square)](http://www.npmjs.org/async-limiter)
  4. [![tests](https://img.shields.io/travis/STRML/async-limiter.svg?style=flat-square&branch=master)](https://travis-ci.org/STRML/async-limiter)
  5. [![coverage](https://img.shields.io/coveralls/STRML/async-limiter.svg?style=flat-square&branch=master)](https://coveralls.io/r/STRML/async-limiter)
  6. This module exports a class `Limiter` that implements some of the `Array` API.
  7. Pass async functions (ones that accept a callback or return a promise) to an instance's additive array methods.
  8. ## Motivation
  9. Certain functions, like `zlib`, have [undesirable behavior](https://github.com/nodejs/node/issues/8871#issuecomment-250915913) when
  10. run at infinite concurrency.
  11. In this case, it is actually faster, and takes far less memory, to limit concurrency.
  12. This module should do the absolute minimum work necessary to queue up functions. PRs are welcome that would
  13. make this module faster or lighter, but new functionality is not desired.
  14. Style should confirm to nodejs/node style.
  15. ## Example
  16. ``` javascript
  17. var Limiter = require('async-limiter')
  18. var t = new Limiter({concurrency: 2});
  19. var results = []
  20. // add jobs using the familiar Array API
  21. t.push(function (cb) {
  22. results.push('two')
  23. cb()
  24. })
  25. t.push(
  26. function (cb) {
  27. results.push('four')
  28. cb()
  29. },
  30. function (cb) {
  31. results.push('five')
  32. cb()
  33. }
  34. )
  35. t.unshift(function (cb) {
  36. results.push('one')
  37. cb()
  38. })
  39. t.splice(2, 0, function (cb) {
  40. results.push('three')
  41. cb()
  42. })
  43. // Jobs run automatically. If you want a callback when all are done,
  44. // call 'onDone()'.
  45. t.onDone(function () {
  46. console.log('all done:', results)
  47. })
  48. ```
  49. ## Zlib Example
  50. ```js
  51. const zlib = require('zlib');
  52. const Limiter = require('async-limiter');
  53. const message = {some: "data"};
  54. const payload = new Buffer(JSON.stringify(message));
  55. // Try with different concurrency values to see how this actually
  56. // slows significantly with higher concurrency!
  57. //
  58. // 5: 1398.607ms
  59. // 10: 1375.668ms
  60. // Infinity: 4423.300ms
  61. //
  62. const t = new Limiter({concurrency: 5});
  63. function deflate(payload, cb) {
  64. t.push(function(done) {
  65. zlib.deflate(payload, function(err, buffer) {
  66. done();
  67. cb(err, buffer);
  68. });
  69. });
  70. }
  71. console.time('deflate');
  72. for(let i = 0; i < 30000; ++i) {
  73. deflate(payload, function (err, buffer) {});
  74. }
  75. t.onDone(function() {
  76. console.timeEnd('deflate');
  77. });
  78. ```
  79. ## Install
  80. `npm install async-limiter`
  81. ## Test
  82. `npm test`
  83. ## API
  84. ### `var t = new Limiter([opts])`
  85. Constructor. `opts` may contain inital values for:
  86. * `t.concurrency`
  87. ## Instance methods
  88. ### `t.onDone(fn)`
  89. `fn` will be called once and only once, when the queue is empty.
  90. ## Instance methods mixed in from `Array`
  91. Mozilla has docs on how these methods work [here](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array).
  92. ### `t.push(element1, ..., elementN)`
  93. ### `t.unshift(element1, ..., elementN)`
  94. ### `t.splice(index , howMany[, element1[, ...[, elementN]]])`
  95. ## Properties
  96. ### `t.concurrency`
  97. Max number of jobs the queue should process concurrently, defaults to `Infinity`.
  98. ### `t.length`
  99. Jobs pending + jobs to process (readonly).