Jekyll2022-02-24T14:37:01+00:00/feed.xmlLandshark.ioDontBeEvil.rip : Search, for Programmers2022-02-17T00:00:00+00:002022-02-17T00:00:00+00:00/2022/02/17/search-for-programmers<p>I’m happy to announce the first agressively alpha release of DontBeEvil.rip, a search engine built for programmers.</p>
<p>DontBeEvil.rip is a year long experiment to see if a small team can build a developer-focused search engine that is self-sustaining on $10 monthly subscriptions. If the service isn’t self-sustaining by March 1st 2023, it will be shut down. If it is, then it will be scaled up in these wonderful ways:</p>
<ul>
<li>Allow groups of subject matter experts to manage site crawl lists</li>
<li>Expand scope of indexed resources to all STEM subjects</li>
<li>Run our own crawler on the most active sites</li>
<li>Add a Way Back Machine style cache</li>
</ul>
<h2 id="status">Status</h2>
<p>HackerNews, StackOverflow, Arxiv abstracts, 2M Github repos, and Programmer Reddit (up to 2020) are being indexed.</p>
<p>The limited, but awesome, features in this first release are:</p>
<ul>
<li>Expressions! Experience the power of Elasticsearch’s <a href="https://www. elastic.co/guide/en/elasticsearch/reference/current/query-dsl-simple-query-string-query.html">Simple Query Strings</a></li>
<li>REST API. For instructions, see the Using It heading below.</li>
</ul>
<h2 id="caveats">Caveats</h2>
<p>I believe that, when you consume a service, if you aren’t paying then you’re being sold. I don’t want to ever be in a position to have to sell my user base. That’s why I’m requiring $10 monthly subscriptions, with a 15 day free trial, from March 1st.</p>
<p>DontBeEvil.rip is aggressively alpha at this point. I’ll do everything I can to ensure 100% uptime, but it <em>will</em> disappear from time to time.</p>
<h2 id="using-it">Using It</h2>
<p>The tl;dr is</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code>apt <span class="nb">install </span>curl jq
pip3 <span class="nb">install </span>jtbl
curl <span class="nt">-O</span> https://github.com/alangibson/dontbeevil/src/rip <span class="o">&&</span> <span class="nb">chmod </span>u+x rip
./rip <span class="s1">'what is a monad'</span>
</code></pre></div></div>
<p><img src="/assets/2022-02-17-search-for-programmers/search_result_screenshot.png" alt="Screenshot of search results" /></p>
<p>If you want to hit the API directly, just take a look at the CLI script. It’s pretty self explanatory.</p>
<div class="language-bash highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span class="nb">cat </span>rip
</code></pre></div></div>
<h2 id="up-next">Up Next</h2>
<p>My next priorities are:</p>
<ul>
<li>Fresher Reddit posts</li>
<li>More Github repos</li>
<li>Index sites on the open web (starting March 2022)</li>
<li>Pagination</li>
<li>Simple web UI</li>
<li>Start charging</li>
</ul>
<p>I’d love to hear any suggestions, questions, comments or ideas you have. My contact info is on the left hand nav bar, so give me a shout.</p>I’m happy to announce the first agressively alpha release of DontBeEvil.rip, a search engine built for programmers.Locating Humans with DNS2021-02-04T00:00:00+00:002021-02-04T00:00:00+00:00/2021/02/04/dns-for-humans<h1 id="locating-humans-with-dns">Locating Humans with DNS</h1>
<p>I recently lost access to a free email account I’ve had since before I can remember (though only temporarily, thank the lord). When it happened, I instantly became effectively uncontactable by a large portion of humanity. My own temporary disappearance from reality started me to thinking about how we can reuse existing infrastructure to store contact information in a durable and highly available way.</p>
<h2 id="searching-for-a-solution">Searching for a solution</h2>
<p>My requirements for a durable contact storage solution are (in no particular order):</p>
<ol>
<li>Public: No authentication shoudl be required to read someone’s contact information.</li>
<li>Globally accessible: Anyone should be able to read the information at any time, from anywhere</li>
<li>Highly available: It should (almost) never go down. One company going bankrupt or changing strategic direction shouldn’t kill it.</li>
<li>Censorship resistant: It should only be possible to take down records with a court order. Corporate censorship should be impossible, or at least historically infrequent.</li>
<li>Cheap: But not free! Free services come with few no guarantees, and even fewer rights for their users.</li>
<li>Low maintenance: User only needs to get involved when the data needs to change. No operational burden</li>
<li>Easy to use: Clients should be able to read information trivially.</li>
</ol>
<p>There are a lot of services that check some of these boxes. Twitter for instance does pretty well, but fails badly on 3, 4 & 5. This is where most services fall down, so I focused there. After pondering this for a while, I realized that we’ve already got a great solution that checks all the boxes: DNS.</p>
<h3 id="highly-available">Highly available</h3>
<p>There are many services that are highly available in the technical sense, meaning they rarely crash, but suffer from economic risk. The ultimate crash is the business that provides the service going bankrupt or changing strategic direction. DNS doesn’t suffer from the same economic risk that centralized services do because there are many, many independent registrars and DNS service providers out there.</p>
<h3 id="censorship-resistance">Censorship resistance</h3>
<p>I would argue that DNS registries are censorship <em>resistant</em> based on the historical record. Whatever you think about the situation, the fact is Parler the application got vaporized in a day, but Parler.com is still registered. The Pirate Bay is one of the most taken down sites in history, and thepiratebay.org is still registered.</p>
<h3 id="cheap">Cheap</h3>
<p>This one is easy. There are loads of registrars that sell a variety of gTLD subdomains, complete with DNS service, for less than $10 per year. Enough said.</p>
<h3 id="easy-to-use">Easy to use</h3>
<p>DNS does so well on the other requirements, it’s a shame that we’d have to use some arcane protocol to query it. Oh wait…</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>curl -s -H 'accept: application/dns-json' 'https://cloudflare-dns.com/dns-query?name=landshark.io&type=TXT' | jq -r .Answer[].data
"v=me; email=alan.gibson@gmail.com"
"v=me; twitter=handheld"
"v=me; blog=https://landshark.io"
"v=me; gravatar=https://www.gravatar.com/avatar/123456"
"v=me; bitcoin=123456789"
</code></pre></div></div>
<p>DNS over HTTP (DoH) means DNS is now trivially easy to query and hard to snoop on. Not too shabby.</p>
<h3 id="a-better-way">A better way</h3>
<p>On the Internet, I’m alan.gibson@gmail.com, as well as twitter.com/handheld, news.ycombinator.com/user?id=alangibson, and so many more. Wouldn’t it be alot simpler if we used DNS TXT records to store all those various identifiers in one place so that you could find me anywhere just by knowing that I’m landshark.io?</p>
<p>Sounds like a pretty slick solution to me. What do you think? <a href="https://news.ycombinator.com/item?id=26024845">Discussion is happening in this Hacker News thread.</a></p>Locating Humans with DNSSaaS Needs a Single Point of Purchase2020-11-13T00:00:00+00:002020-11-13T00:00:00+00:00/2020/11/13/saas-needs-a-single-point-of-purchase<p>The unassailable advantage that big cloud providers like AWS and Azure have over the rest of the SaaS industry isn’t their quality or pace of innovation; it’s that they’re a single point of purchase for a variety of services. If there was a single vendor that large corporations could contract with and pay for SaaS, the big cloud providers market share would have hit its peak.</p>
<p>I can’t overstate the importance of purchasing mechanics in the decision of what SaaS to use inside large corporations. Because of the arcane and complex purchasing requirements you run into, the simple act of buying software is incredibly difficult and time consuming. People who’ve never worked in a BigCo often don’t realize what an unspeakable nightmare it can be. Because, you see, one does not simply put it on the credit card.</p>
<h2 id="a-series-of-unfortunate-events">A Series of Unfortunate Events</h2>
<p>I once spent <em>5 months</em> licensing Jira. We first had to get a quote, even though it’s fixed price, because Purchasing’s rules said you have to have a quote. After we got the quote, we realized we had no way to pay Atlassian because they don’t take POs and we didn’t pay any way but PO. Putting software on your corporate credit card was explicitly forbidden. So we had to engage a reseller to act as an intermediary for no other reason that they could pay with a credit card. Of course they then had to get another quote that eventually expired due to their general incompetence. Rinse and repeat for several more months until we finally got access.</p>
<p>We never licensed another SaaS product, though we had plenty of budget for it. Anything we need, we made it work on AWS tough we’d rather not have the maintenance hassle. We use open source when we’d be willing and able to license a tool simply because no one can bring themselves to relive the Jira nightmare.</p>
<h2 id="a-solution">A Solution</h2>
<p>The SaaS industry needs is a go-to biller where teams inside large corporations could license software under a standing PO. To the company, the entire SaaS industry would then look like a single vendor with many products for sale (just like AWS). Teams would no longer have to think long and hard about whether some new SaaS tool was worth putting the effort in to license, because I can tell you the answer usually ends up being ‘Buh, just make it work with AWS.’</p>
<p>I know there are already aggregators like App Sumo. However, they tend to be based more around bargains. Large companies will certainly take a deal, but it’s not a major driver for them. We paid that reseller I mentioned an extra <em>25%</em> just to use their Visa card.</p>
<p>Obviously this is a big ask. It would require a lot of major players to sign on before even launching. Maybe a non-profit SaaS consortium could make it work. What do you think? Madness or genius?</p>
<p><a href="https://news.ycombinator.com/item?id=25081711">Hacker News Discussion Thread</a></p>The unassailable advantage that big cloud providers like AWS and Azure have over the rest of the SaaS industry isn’t their quality or pace of innovation; it’s that they’re a single point of purchase for a variety of services. If there was a single vendor that large corporations could contract with and pay for SaaS, the big cloud providers market share would have hit its peak.No Free Features2020-11-09T00:00:00+00:002020-11-09T00:00:00+00:00/2020/11/09/no-free-features<p><a href="https://news.ycombinator.com/item?id=25032105">This thread recently hit the top of Hacker news</a>.</p>
<blockquote>
<p>No More Free Work from Marak: Pay Me or Fork This</p>
<p>Respectfully, I am no longer going to support Fortune 500s
( and other smaller sized companies ) with my free work.</p>
</blockquote>
<p>The gist is that Marak, who’s on the brink of homelessness after his apartment building caught fire, is no longer interested in doing unpaid work for businesses using his <a href="https://github.com/Marak/faker.js">faker.js</a> project. He seems to be getting a lot of support from the open source developer community.</p>
<h2 id="unpaid-interns">Unpaid Interns</h2>
<p>The foundational principle of open source is “fix your problem, then give the world a copy of the solution.” So let’s get one thing straight: <em>open source developers are not volunteering to fix your problem.</em> They are fixing their own problems, then letting you use the solution too because it costs them nothing. That near-zero cost of replicating software is why open source works.</p>
<p>Because of this, I don’t think developers claiming to do open source should expect compensation for features that they need for themselves. But developing a new feature that they don’t need is something different entirely. In IT we call that a Change Request, and CRs come with a fee to cover them. ‘Near-zero cost’ doesn’t apply anymore because now they’re taking on a lot of work they otherwise wouldn’t have done.</p>
<p>Not recognizing this difference has led to a situation where for-profit entities are using open-source devs as unpaid interns. Well it’s worse really: at least interns get resume filler.</p>
<h2 id="no-more-free-features">No More Free Features</h2>
<p>I look forward to a day when asking anyone to do unpaid labor is considered unethical by our industry. That goes for feature requests on open source projects, on unpaid internships, and on unpaid ‘take home’ interview assignments.</p>
<p>Requesting work in an economic context without offering compensation in some form is morally indefensible. It’s wrong because unpaid labor is wrong. It’s wrong because presuming on anyone’s helpful nature is wrong. We shouldn’t be using bounties to move our change requests to the head of the line because we shouldn’t even be making requests without a bounty attached.</p>
<p>(<a href="https://news.ycombinator.com/item?id=25034809">Official Hacker News discussion thread</a>)</p>This thread recently hit the top of Hacker news.Please Stop Linking To Google Colaboratory2020-10-21T00:00:00+00:002020-10-21T00:00:00+00:00/2020/10/21/stop-linking-to-google-colaboratory<p>I’m seeing more and more machine learning resources linking to Google Colaboratory. Please, for the sake of future learners, don’t do this. If you want to host your examples on Google Colab, save yourself some effort and all of us some heartache, and just go ahead and delete it before Google inevitably does.</p>
<p>Now if you want to use it for your own purposes in the short time it will be alive, go right ahead. Just don’t make us trying to learn from your hard work (and we thank you for it) suffer when they inevitably kill it.</p>
<p>Here’s problem number 1:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>https://colab.research.google.com
</code></pre></div></div>
<p>Colaboratory is a Google Research project. If you thought Google proper killed projects fast and often, how commited to you think Google Research is to keeping services alive over the long term?</p>
<p>Here’s problem number 2:</p>
<p>“Colab is a hosted Jupyter notebook service that requires no setup to use, while providing free access to computing resources including GPUs.”</p>
<p>(Source)[https://research.google.com/colaboratory/faq.html]</p>
<p>Free access. To GPUs. How long do you think they will keep paying that bill for you?</p>
<p>Here’s problem number 3:</p>
<p>“The Azure Notebooks preview will be retired on January 15th, 2021, and all user data will be destroyed. Please download your user data before then.
To execute notebooks or create new notebook content, learn about our other notebooks experiences at Microsoft.”</p>
<p>(Source)[https://notebooks.azure.com/]</p>
<p>It’s already happened before.</p>
<h2 id="alternatives">Alternatives</h2>
<p>Azure Notebooks and Google Colaboratory are both glorified Jupyter Notebook servers, so let’s think about how we can get the same effect without the link rot. If you are just blogging, consider statically rendering your Jupyter Notebook as HTML with nbconvert. If you want live examples, consider running your own Jupyter Notebook server. It will cost you something, but it will also protect the value of your work for as long as <em>you</em> feel like it should be available.</p>
<p>And finally, thanks for the work you’re doing to make the world a smarter place.</p>I’m seeing more and more machine learning resources linking to Google Colaboratory. Please, for the sake of future learners, don’t do this. If you want to host your examples on Google Colab, save yourself some effort and all of us some heartache, and just go ahead and delete it before Google inevitably does.You Can Compete With Free When It’s B2B2020-10-20T00:00:00+00:002020-10-20T00:00:00+00:00/2020/10/20/you-can-compete-with-free<p>Retool has announced that it raised $50M from Sequoia only 2 years after launch for what is essentially a front-end builder. While explaining to a non-developer friend how you can build a business selling something that countless web developers can cobble together out of free open source components, I accidentally busted this rhyme:</p>
<p>You can compete with free,
when it comes to B2B,
because for businesses
no-charge isn’t free</p>
<p>Look at it from a business’s point of view. Let’s say they pay a web developer $100 / hour. If it takes that dev 10 hours to piece together a front end from React, Bootstrap, and several tends of thousands of Javascript packages, that app cost them $1000. The fact that they don’t pay for the open source components is irrelevant.</p>
<p>Now imagine they could get the same app with 2 hours of effort using a tool that costs $100 / month. That $100 tool ends up costing them $-700. Paying up starts to sound like a pretty good deal</p>
<p>There are lots of examples of this economic story playing out if you look for them. Any development team can replicate the essential elements of Algolia with Apache Solr or Elasticsearch, but Algolia is going strong. Open source CMS options abound, but Contentful raised an $80M series E recently.</p>
<p>The big caveat here is that this <em>does not</em> apply in the B2C world. Consumers are famously blind to total cost of ownership, both monetary and intangible (like giving up any semblance of privacy for a ‘free’ navigation app). Free really is nigh on impossible to beat when you are targeting consumers. And the false economy of cheap-but-fragile junk is so compelling that finding <a href="https://www.reddit.com/r/BuyItForLife">things that won’t break in a couple of years is an armchair sport</a>.</p>
<p>Next time your trying to think of a business idea, ask yourself this question:</p>
<blockquote>
<p>What are businesses dependent on that they aren’t paying for directly but they are paying dearly for in employee hours?</p>
</blockquote>
<p>Whatever answer you come up with, it’s a million dollar idea.</p>Retool has announced that it raised $50M from Sequoia only 2 years after launch for what is essentially a front-end builder. While explaining to a non-developer friend how you can build a business selling something that countless web developers can cobble together out of free open source components, I accidentally busted this rhyme:Google One blocks transnational families2020-09-06T00:00:00+00:002020-09-06T00:00:00+00:00/2020/09/06/google-one-blocks-transnational-families<p>Here’s one for the Unfortunate Error Message hall of fame.</p>
<p><img src="/assets/2020-09-06-google-one-bans-separated-families/no-family-for-you.png" width="400" /></p>
<p>I got this while trying to add my wife (who is not from the USA) to my Google One storage plan. Apparently Google never counted on family members living in different countries.</p>
<p>I see this as a rather embarassing example of Conway’s Law. Our family structure isn’t compatible with the structure of Google’s business, so no family plan for us.</p>
<p>Hey Google: things are complicated outside of Mountain View. Try putting a little more effort into covering us edge cases. Don’t make us pay for your convenience.</p>Here’s one for the Unfortunate Error Message hall of fame.Web by Google (TM)2020-08-16T00:00:00+00:002020-08-16T00:00:00+00:00/2020/08/16/web-by-google<p>Looking at Mozilla’s finances, it’s reasonable to conclude that Google is keeping them on life support to keep the anti-trust hounds at bay. Mozilla’s deal with Google will account for at least 70% of their revenues going forward. That’s over $400 million to be the default search provider in 4% of browsers. For 1 year. I’ll bet my next paycheck that Google reupped this rather generous agreement to avoid a repeat of Microsoft’s troubles over an Internet Explorer monopoly. My guess is that Google will decide it’s in their interest to pull the plug right around the time this latest wave of anti-trust talk blows over.</p>
<p>With the troubles at Mozilla, Google is one step closer to replicating the WeChat and Facebook walled-garden model on the Web. A quick survey a the field of play shows just how far Google has come in capturing the once open Web.</p>
<h2 id="browser-war-is-over">(Browser) War is over</h2>
<p>Chrome’s ability to dictate web standards will only get stronger over time. Safari and Firefox have been able to apply some shame to Chrome on things like disabling third-party cookies, but soon it’ll just be Apple left with a voice.</p>
<p>But Google doesn’t even need Chrome to dictate standards since it controls the Web’s front door. AMP, a technology no one asked for, is now on over 70% of all marketing websites for no other reason than Google said so.</p>
<p>And let’s not forget that most people now access the web via mobile devices, 75% of which run Google’s Android.</p>
<h2 id="original-sins">Original Sins</h2>
<p>Two deficiencies have determined the course the Web has taken: lack of native search and lack of native payments.</p>
<p>Lacking native distributed search allowed Google to grab a monopoly position as the entry point to the web. The Web’s orignal architects were off base on hyperlinks; it turns out people just want to skip right to the answer they’re looking for. There was once the beginnings of an alternative in public bookmarking sites, but social networks came and sucked all the air out of that space.</p>
<p>Lack of a safe and easy way of exchanging funds was a big factor in advertising becoming the dominant business model on the Web. No-charge services became the norm because, for a long time, there was no reasonable way to pay for them. Enter advertising to fill the void. I think it’s safe to say that Google’s AdWords is the dominant advertising platform on the open Web, which means it holds a commanding position in the Web’s finances.</p>
<p>I’m also going to throw in an honorable mention to poor native video support (it is hyper_text_ after all). It’s beating a dead horse to point out that Google also serves up the vast majority of videos streamed on the Web.</p>
<h2 id="so-then">So then…</h2>
<p>So Google controls the Web’s search and video infrastructure. It can and does dictate standards and media formats. It also controls a huge chunk of the revenues available when publishing and selling on the Web. It even controls the operating system and browser through which most people interact with it.</p>
<p>Google’s capture of the Web is a <em>fait accompli</em>. Only legislation will keep the World Wide Web from finally becoming Web by Google (TM).</p>Looking at Mozilla’s finances, it’s reasonable to conclude that Google is keeping them on life support to keep the anti-trust hounds at bay. Mozilla’s deal with Google will account for at least 70% of their revenues going forward. That’s over $400 million to be the default search provider in 4% of browsers. For 1 year. I’ll bet my next paycheck that Google reupped this rather generous agreement to avoid a repeat of Microsoft’s troubles over an Internet Explorer monopoly. My guess is that Google will decide it’s in their interest to pull the plug right around the time this latest wave of anti-trust talk blows over.Do Your Estimates Suck?2020-02-12T00:00:00+00:002020-02-12T00:00:00+00:00/2020/02/12/do-your-estimates-suck<p>Do your estimates suck? You know they probably do, even if you can’t prove it. We all know that estimates are just white lies we use to keep management happy. But do you know how far off they really are?</p>
<h1 id="bitter-truth">Bitter Truth</h1>
<p>I’ve long suspected that the time my team is spending on estimates is wasted, but I haven’t had the numbers to back it up. Fed up after another round of mile-off estimates caused us to nearly miss a delivery date, I calculated the correlation coefficient between our estimates and the actual lead times of our user stories.</p>
<p>With the wealth of information stashed away in Jira, it proved to be straightforward. I came up with a Pearson’s r of 0.28 (to no one’s surprise). Given that the scale goes from 0 to 1, that’s some very weak correlation. Just check out this scatterplot:</p>
<p><img src="/assets/2020-02-12-do-your-estimates-suck/estimate-accuracy-scatterplot.svg" alt="Estimate / Lead Time scatterplot" /></p>
<p>The upshot of this is that now I can prove that we need to increase our estimates across the board. <a href="https://www.youtube.com/watch?v=eisuQefYw_o">As Uncle Bob said</a> (fast forward to 17:00), “everything in software takes 3 times longer than you think it should, even when you know this and take it into account.” It’ll be a hard sell to the team (optimisim is an occupational hazard for programmers after all), but the numbers don’t lie.</p>
<h1 id="a-new-hope">A New Hope</h1>
<p>I’m sure we’re not the only ones in this boat, burning hours and brain cells doing estimates that don’t inform. So I’ve turned this into free and easy to use tool: <a href="https://without.fail/estimate-accuracy">Have at it Jira users.</a>.</p>
<p>I challenge everyone to take me up on my offer and see how much information your estimates are really giving you. Check yourself after every sprint for 3 months and use the observed correlation to get better at estimating.</p>Do your estimates suck? You know they probably do, even if you can’t prove it. We all know that estimates are just white lies we use to keep management happy. But do you know how far off they really are?Fix your development process by visualizing problems2020-01-08T00:00:00+00:002020-01-08T00:00:00+00:00/2020/01/08/andon-for-software-development<p>Finding ways for developers to effectively raise process and quality issues outside of their immediate team is a huge challenge. Often an issue will only make it into the light when it’s been a problem for days or weeks. And the complications only multiply when you’re dealing with large, distributed teams.</p>
<p>I’m talking about problems that bedevil us all, like:</p>
<ul>
<li>Definition of Done not having been met on a Sprint Backlog item</li>
<li>Finding an unclear requirement</li>
<li>A necessary resource not being available</li>
<li>Realizing you need to take on an unacceptable ammount of technical debt to meet a deadline</li>
<li>Discovering that a dependency has a critical CVE filed against it and there’s no fix yet</li>
<li>Someone commits a performance-wrecking change</li>
<li>Just not being able to get the damn thing to run…</li>
</ul>
<h1 id="the-problems">The problem(s)</h1>
<p>Failing to address issues as the occur only allows them to accumulate. In the world of software developments, a few hiccups can turn into terminal brain cancer pretty fast.</p>
<p>I’d rather not recall how many times I’ve overheard this conversation well after the train as gone off the tracks:</p>
<ul>
<li>A: Why am I only hearing about this now?</li>
<li>B: Well I said something at the time.</li>
<li>A: Well you didn’t say anything to me.</li>
<li>B: Well whenever I do say something, no one listens.</li>
<li>A: No, you!</li>
<li>B: No, YOU!</li>
</ul>
<p>Sprints only exacerbate this problem by building in artifical latency. People that see the problem tend to wait until the sprint retrospective to bring it up.</p>
<h1 id="introducing-andon">Introducing Andon</h1>
<p>Missing issues as they are occuring is a huge, costly problem in manufacturing. The Toyota Production System originated the idea of Andon, a visual signaling system that allows everyone to effortlessly highlight issues as they occur. Andon is, at its core, a way of weaving quality into the fabric of a product.</p>
<p>Team members are encouraged to pull a red or yellow chord whenever they see a problem, depending on if the issue is blocking or not. Pulling the yellow chord triggers a troubleshooter to come help with the issue; red shuts down the production line and resets the entire organization’s priority to immedately clearing the blocker. Yellow chord pulls keep quality problems from being propogated down stream where they can accumulate into a red chord pull. Thus Andon focuses teams on fire prevention, not fire fighting. Quality and efficiency become a reflex.</p>
<h1 id="applying-it-to-software-development">Applying it to software development</h1>
<p>So what does this have to do with development teams? Well if you have a CI system that stops on build or test errors and flags the failing job, you’re already using Andon. Quality problems are visually indicated and actively prevented from moving on downstream. I’m here to argue that we as developers should go “full Andon.”</p>
<p>Replace that dashboard full of metrics no one pays attention to with an Andon board. Give each team its own Andon light on the board. Give each member of the team access to the “chord.” Designate a lead for each team who’s priority it is to jump in and help resolve or escalate issues as soon as they are found. Then marvel as newly empowered team members start taking quality issues seriously.</p>
<p>How do we assure that people actually take yellow signals seriously? Won’t we just learn to ignore them and make a joke of the whole system? As developers we love automation, so automate it. Yellow pulls should go red after a set time. Shut down the line by sending an automated Slack message asking everyone on the team to swarm the issue and get things moving again.</p>
<h1 id="getting-better-every-day">Getting better every day</h1>
<p>In Andon, every chord pull is treated as an opportunity for continuous improvement. When an issue is found, the process itself should be patched, either through automation or documentation. This brand of <em>really</em> continuous improvement will evolve your process much faster than addressing a couple of issues after each Sprint retrospective.</p>
<p>Take the challenge to prioritize quality and give Andon a try. Let me know how it goes. :)</p>Finding ways for developers to effectively raise process and quality issues outside of their immediate team is a huge challenge. Often an issue will only make it into the light when it’s been a problem for days or weeks. And the complications only multiply when you’re dealing with large, distributed teams.