Our weekly spotlight of a cool project in the open source community that are looking for exposure or assistance. If you’d like your project to be featured, hit reply and let us know about it.

Project Spotlight: Verk

Verk is a asynchronous job processing system written in Elixir. Its main goal is to ensure that a job is executed no matter what happens. Jobs do not exist in-memory only, so users can safely enqueue jobs and expect them to be run at least once.

image alt text

Verk borrows heavily from Sidekiq/Resque. A client from these projects can enqueue new jobs to Verk, and have them executed in Elixir. This creates a great path for those who are migrating Rails applications to Elixir, or who have mixed deploys and want to scale a single background job queue. Verk has already processed billions of jobs on a single deploy since its conception, and it has its own Verk Web interface written in Phoenix.

Verk instances today are standalone and do not communicate as a cluster. The next big step for the project is ensuring Verk instances can work together knowing what each instance is executing. So for example pausing a specific queue should pause for every instance and scaling up and down should automatically capture lost jobs etc.

Being able to work as a standalone is a boon to those deploying to services like Heroku, where Erlang clustering isn’t feasible.

Check out the open issues and jump in.

$ tail /dev/random

# it’s not news, but it is newsworthy

Hacker News Tools of the Trade - a collection of tools used by Hacker News, AngelList, and Quora users in their startups. It is…extensive.

pigeon-maps - React Maps without external dependencies - "a performance-first React-centric extendable map engine"

Project Python - "a free interactive book that will teach you to code in Python, using graphics, animations, and games."

jupytext - Jupyter notebooks as Markdown documents, Julia, Python or R scripts

An Engineer's Guide to the Docuverse - A very detailed look into how Project Xanadu -- the original and ongoing hypertext project from Ted Nelson -- works.

React Virtualized - efficiently render large lists and tabular data in React

Introspected REST - I don’t agree with all of the assertions made here, but it would be nice to hear from someone who tried this out on a real project.

refract.js - isolated side effects for React / Redux projects. This looks like it could solve one of my biggest complaints in React apps.

Announcing Gatsby 2.0 - Gatsby’s a great static PWA generator. Highlights for 2.0: Reduced build times ~75%; shrunk JS runtime by 31%; webpack 4, Babel 7, React 16.5 updates.

Papers with code - "Papers with code. Sorted by stars. Updated weekly." No bullshit, links directly to PDFs. Lots of cool image ML stuff, be sure to check out ‘Learning to See in the Dark.’ TRIGGER WARNING: Actual sorcery.

Josh Adams

I've been building web-based software for businesses for over 18 years. In the last four years I realized that functional programming was in fact amazing, and have been pretty eager since then to help people build software better.

  1. Comments for Spotlight: Verk - background job processing

You must login to comment

You May Also Like