mirror of
https://git.sr.ht/~seirdy/seirdy.one
synced 2024-11-23 21:02:09 +00:00
More CSS containment, adopt shortcodes
- Adopt more shortcodes in older posts. - Contain figures, excluding images. Slightly decreases paint times. - Fix spacing issues for nested articles. - Always enable vertical scrollbar, since pretty much all pages are taller than the viewport. Eliminates a layout shift. - Moar microdata - Set fixed updated dates for some posts so they don't get new date-updated values until I actually change the content significantly
This commit is contained in:
Rohan Kumar
parent
8f4e1b7468
commit
a96b77793b
15 changed files with 341 additions and 96 deletions
|
|
@ -10,9 +10,9 @@
|
||||||
* don't actually use those for styling.
|
* don't actually use those for styling.
|
||||||
*
|
*
|
||||||
* To keep myself from caring about minute details, I limited myself to
|
* To keep myself from caring about minute details, I limited myself to
|
||||||
* only defining spacing in increments of .125em. Borders are either
|
* only defining spacing in increments of .125em. Pixels are 1px or
|
||||||
* 1px or 3px; percentages are in increments of 2.5%. This also improves
|
* multiples of 3px; percentages are in increments of 2.5%. This also
|
||||||
* compression. No more "finding the perfect value".
|
* improves compression. No more "finding the perfect value".
|
||||||
*
|
*
|
||||||
* I cite WCAG 2.2 success criteria with "SC". I also tried to meet
|
* I cite WCAG 2.2 success criteria with "SC". I also tried to meet
|
||||||
* the Google a11y requirement of 48px tap targets separated by atl
|
* the Google a11y requirement of 48px tap targets separated by atl
|
||||||
|
|
@ -26,6 +26,9 @@ html {
|
||||||
* <100dpi screens: sans-serif is better. Why did browsers settle
|
* <100dpi screens: sans-serif is better. Why did browsers settle
|
||||||
* on serif being the default?? */
|
* on serif being the default?? */
|
||||||
font: 100%/1.5 sans-serif;
|
font: 100%/1.5 sans-serif;
|
||||||
|
/* Nearly every page on my site is taller than the viewport.
|
||||||
|
* Paint the scrollbar ASAP to prevent layout shifts. */
|
||||||
|
overflow-y: scroll;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* This should not take effect on printouts, to save paper. */
|
/* This should not take effect on printouts, to save paper. */
|
||||||
|
|
@ -74,29 +77,20 @@ html {
|
||||||
|
|
||||||
/* Compensate for misalignment and wasted space caused by padding
|
/* Compensate for misalignment and wasted space caused by padding
|
||||||
* and margin adjustments in nav children made to meet SC 2.5.5 */
|
* and margin adjustments in nav children made to meet SC 2.5.5 */
|
||||||
header nav,
|
header > nav,
|
||||||
footer nav {
|
footer > nav {
|
||||||
margin: 0 0 -.75em -.25em;
|
margin: 0 0 -.75em -.25em;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* Google a11y: ensure tap targets have >=8px space between them */
|
summary {
|
||||||
nav li,
|
|
||||||
details li {
|
|
||||||
padding: 1em 0;
|
|
||||||
}
|
|
||||||
|
|
||||||
/* SC 2.5.5: Increase nav link tap target size a bit */
|
|
||||||
nav a,
|
|
||||||
summary,
|
|
||||||
details li a {
|
|
||||||
padding: .875em .25em;
|
|
||||||
}
|
|
||||||
|
|
||||||
details summary {
|
|
||||||
/* The tappable region of a <summary> extends across the page.
|
/* The tappable region of a <summary> extends across the page.
|
||||||
* we need to tell users that the full space is interactive.
|
* we need to tell users that the full space is interactive.
|
||||||
* Use a border for that. */
|
* Use a border for that. */
|
||||||
border: 1px solid #999;
|
border: 1px solid #999;
|
||||||
|
/* It's not obvious that a <summary> has button semantics.
|
||||||
|
* "cursor: pointer" is used on MDN, web.dev, GitHub, gov.uk, and
|
||||||
|
* others so it's not "novel" and won't surprise users. */
|
||||||
|
cursor: pointer;
|
||||||
}
|
}
|
||||||
|
|
||||||
/* If we have a list of disclosure widgets, we need some
|
/* If we have a list of disclosure widgets, we need some
|
||||||
|
|
@ -105,6 +99,18 @@ html {
|
||||||
margin: .5em .25em;
|
margin: .5em .25em;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Google a11y: ensure tap targets have >=8px space between them */
|
||||||
|
nav li,
|
||||||
|
li details li {
|
||||||
|
padding: 1em 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
/* SC 2.5.5, Google a11y: Increase tap target size a bit */
|
||||||
|
summary,
|
||||||
|
li > a {
|
||||||
|
padding: .875em .25em;
|
||||||
|
}
|
||||||
|
|
||||||
nav li ol {
|
nav li ol {
|
||||||
/* Don't duplicate bottom space: the last list item in the nested
|
/* Don't duplicate bottom space: the last list item in the nested
|
||||||
* list and the list item that is the entire nested list will both
|
* list and the list item that is the entire nested list will both
|
||||||
|
|
@ -192,13 +198,22 @@ kbd {
|
||||||
|
|
||||||
/* narrow screens: remove unused figure margins
|
/* narrow screens: remove unused figure margins
|
||||||
* figure captions shouldn't look like regular paragraphs; there should
|
* figure captions shouldn't look like regular paragraphs; there should
|
||||||
* be some extra space. */
|
* be some extra space.
|
||||||
|
* This does not hold true for figures in nested articles; the nested
|
||||||
|
* article should instead get the spacing. */
|
||||||
|
article,
|
||||||
figure {
|
figure {
|
||||||
margin: 1.5em 0;
|
margin: 1.5em 0;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
div[itemprop="articleBody"] > article > figure {
|
||||||
|
margin: 0;
|
||||||
|
}
|
||||||
|
|
||||||
/* figcaptions should be distinct from surrounding paragraphs tho */
|
/* figcaptions should be distinct from surrounding paragraphs tho */
|
||||||
figure:not([itemtype]) figcaption {
|
article[itemtype="https://schema.org/ImageObject"] figcaption,
|
||||||
|
figure[itemtype="https://schema.org/ImageObject"] figcaption {
|
||||||
|
contain: content;
|
||||||
margin: 0 10%;
|
margin: 0 10%;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -207,6 +222,7 @@ figure:not([itemtype]) figcaption {
|
||||||
* font size preferences. */
|
* font size preferences. */
|
||||||
code,
|
code,
|
||||||
kbd,
|
kbd,
|
||||||
|
pre, /* Needed for ancient browsers */
|
||||||
samp {
|
samp {
|
||||||
font-family: monospace, monospace;
|
font-family: monospace, monospace;
|
||||||
}
|
}
|
||||||
|
|
@ -251,7 +267,8 @@ pre {
|
||||||
/* center standalone images; same justification as
|
/* center standalone images; same justification as
|
||||||
* for centering the body contents. Also makes images easier to see
|
* for centering the body contents. Also makes images easier to see
|
||||||
* for people holding a device with one hand. */
|
* for people holding a device with one hand. */
|
||||||
picture img {
|
picture > img {
|
||||||
|
/* Don't decode the image until it's near the fold */
|
||||||
display: block;
|
display: block;
|
||||||
height: auto;
|
height: auto;
|
||||||
margin: auto;
|
margin: auto;
|
||||||
|
|
@ -274,6 +291,12 @@ section[role="doc-endnotes"] {
|
||||||
content-visibility: auto;
|
content-visibility: auto;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/* Contain figures that are not images. */
|
||||||
|
:not(div[itemprop="articleBody"]) > :not(article) > figure:not([itemtype="https://schema.org/ImageObject"]) {
|
||||||
|
contain: content;
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
/* Some browser focus indicators are inaccessible; override them with
|
/* Some browser focus indicators are inaccessible; override them with
|
||||||
* something more visible. See WCAG 2.x SC 2.4.12. I also tried to
|
* something more visible. See WCAG 2.x SC 2.4.12. I also tried to
|
||||||
* match browser behavior; mainstream browsers use :focus-visible
|
* match browser behavior; mainstream browsers use :focus-visible
|
||||||
|
|
|
||||||
|
|
@ -20,6 +20,9 @@ copyright = "Copyright © 2021 Rohan Kumar"
|
||||||
dark = "auto"
|
dark = "auto"
|
||||||
highlight = false
|
highlight = false
|
||||||
|
|
||||||
|
[frontmatter]
|
||||||
|
lastmod = ['lastmod', ':git', 'date', 'publishDate']
|
||||||
|
|
||||||
[author]
|
[author]
|
||||||
name = "Rohan Kumar"
|
name = "Rohan Kumar"
|
||||||
first = "Rohan"
|
first = "Rohan"
|
||||||
|
|
|
||||||
|
|
@ -19,7 +19,7 @@ This page also exists on the [tildeverse](https://tildeverse.org/), a bunch of \
|
||||||
Content on this site also appears on <a href="gemini://seirdy.one">my Gemini capsule</a>
|
Content on this site also appears on <a href="gemini://seirdy.one">my Gemini capsule</a>
|
||||||
|
|
||||||
Location (Rohan, meat­space) {#location-rohan-meatspace}
|
Location (Rohan, meat­space) {#location-rohan-meatspace}
|
||||||
---------------------------
|
--------------------------------
|
||||||
|
|
||||||
Currently living at home in Cupertino, CA
|
Currently living at home in Cupertino, CA
|
||||||
|
|
||||||
|
|
@ -29,10 +29,15 @@ Location (Seirdy, online)
|
||||||
My handle is "Seirdy" on all the platforms I use.
|
My handle is "Seirdy" on all the platforms I use.
|
||||||
|
|
||||||
- My [tildeverse page](https://envs.net/~seirdy/).
|
- My [tildeverse page](https://envs.net/~seirdy/).
|
||||||
|
|
||||||
- Software forges: [Sourcehut](https://sr.ht/~seirdy), [GitHub](https://github.com/Seirdy), [GitLab](https://gitlab.com/Seirdy), and [Codeberg](https://codeberg.org/Seirdy).
|
- Software forges: [Sourcehut](https://sr.ht/~seirdy), [GitHub](https://github.com/Seirdy), [GitLab](https://gitlab.com/Seirdy), and [Codeberg](https://codeberg.org/Seirdy).
|
||||||
|
|
||||||
- Social (federated): I'm [@Seirdy<wbr>@pleroma<wbr>.envs.net](https://pleroma.envs.net/seirdy) on the Fediverse.
|
- Social (federated): I'm [@Seirdy<wbr>@pleroma<wbr>.envs.net](https://pleroma.envs.net/seirdy) on the Fediverse.
|
||||||
|
|
||||||
- Social (non-federated): I'm Seirdy on [Hacker News](https://news.ycombinator.com/user?id=Seirdy), [Lobsters](https://lobste.rs/u/Seirdy), [Reddit](https://www.reddit.com/user/Seirdy/), [Tildes.net](https://tildes.net/user/Seirdy), and Linux Weekly News.
|
- Social (non-federated): I'm Seirdy on [Hacker News](https://news.ycombinator.com/user?id=Seirdy), [Lobsters](https://lobste.rs/u/Seirdy), [Reddit](https://www.reddit.com/user/Seirdy/), [Tildes.net](https://tildes.net/user/Seirdy), and Linux Weekly News.
|
||||||
|
|
||||||
- Email: my address is [seirdy<wbr>@seirdy.one](mailto:seirdy@seirdy.one). I typically sign my emails with my public PGP key: [`1E892DB2A5F84479`](./publickey.asc). My key is also available via WKD.
|
- Email: my address is [seirdy<wbr>@seirdy.one](mailto:seirdy@seirdy.one). I typically sign my emails with my public PGP key: [`1E892DB2A5F84479`](./publickey.asc). My key is also available via WKD.
|
||||||
|
|
||||||
- Chat: for IRC, my nick is Seirdy on Libera.chat, Snoonet, OFTC, Tilde.Chat, and a few smaller networks. I'm also [@seirdy<wbr>:seirdy.one](https://matrix.to/#/@seirdy:seirdy.one) on Matrix.
|
- Chat: for IRC, my nick is Seirdy on Libera.chat, Snoonet, OFTC, Tilde.Chat, and a few smaller networks. I'm also [@seirdy<wbr>:seirdy.one](https://matrix.to/#/@seirdy:seirdy.one) on Matrix.
|
||||||
|
|
||||||
My secondary Matrix account for Synapse-only rooms is `@seirdy:fairydust.space`. My Matrix account used to be `@seirdy:envs.net` but I've since migrated to my own Conduit server.
|
My secondary Matrix account for Synapse-only rooms is `@seirdy:fairydust.space`. My Matrix account used to be `@seirdy:envs.net` but I've since migrated to my own Conduit server.
|
||||||
|
|
|
||||||
|
|
@ -1,3 +1,5 @@
|
||||||
|
## Background
|
||||||
|
|
||||||
I find it easy to handle views different from my own. I feel more troubled when I see people agree with me for the wrong reasons.
|
I find it easy to handle views different from my own. I feel more troubled when I see people agree with me for the wrong reasons.
|
||||||
|
|
||||||
It's no secret that I'm a passionate supporter of software freedom: I've written two posts about how Free, Libre, and Open-Source Software (FLOSS) is necessary but insufficient to preserve user autonomy:
|
It's no secret that I'm a passionate supporter of software freedom: I've written two posts about how Free, Libre, and Open-Source Software (FLOSS) is necessary but insufficient to preserve user autonomy:
|
||||||
|
|
@ -7,6 +9,8 @@ It's no secret that I'm a passionate supporter of software freedom: I've written
|
||||||
|
|
||||||
After two posts spanning over 5000 words, I need to add some nuance.
|
After two posts spanning over 5000 words, I need to add some nuance.
|
||||||
|
|
||||||
|
## Introduction
|
||||||
|
|
||||||
One of the biggest parts of the Free and Open Source Software definitions is the freedom to study a program and modify it; in other words, access to editable source code. I agree that such access is essential; however, far too many people support source availability for the *wrong* reasons. One such reason is that source code is necessary to have any degree of transparency into how a piece of software operates, and is therefore necessary to determine if it is at all secure or trustworthy. Although security through obscurity is certainly not a robust measure, this claim has two issues:
|
One of the biggest parts of the Free and Open Source Software definitions is the freedom to study a program and modify it; in other words, access to editable source code. I agree that such access is essential; however, far too many people support source availability for the *wrong* reasons. One such reason is that source code is necessary to have any degree of transparency into how a piece of software operates, and is therefore necessary to determine if it is at all secure or trustworthy. Although security through obscurity is certainly not a robust measure, this claim has two issues:
|
||||||
|
|
||||||
* Source code describes what a program is designed to do; it is unnecessary and insufficient to determine if what it actually does aligns with its intended design.
|
* Source code describes what a program is designed to do; it is unnecessary and insufficient to determine if what it actually does aligns with its intended design.
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,6 @@
|
||||||
---
|
---
|
||||||
date: "2022-02-02T23:16:00+00:00"
|
date: "2022-02-02T23:16:00+00:00"
|
||||||
|
lastmod: "2022-03-17T23:32:15-07:00"
|
||||||
description: "While source code is critical for user autonomy, it isn't required to evaluate software security or understand run-time behavior."
|
description: "While source code is critical for user autonomy, it isn't required to evaluate software security or understand run-time behavior."
|
||||||
outputs:
|
outputs:
|
||||||
- html
|
- html
|
||||||
|
|
@ -8,15 +9,31 @@ footnote_heading: Notes
|
||||||
featured: 4
|
featured: 4
|
||||||
title: "The right thing for the wrong reasons: FLOSS doesn't imply security"
|
title: "The right thing for the wrong reasons: FLOSS doesn't imply security"
|
||||||
---
|
---
|
||||||
|
<section role="doc-preface" itemprop="backstory">
|
||||||
|
|
||||||
|
<h2 id="background">Background</h2>
|
||||||
|
|
||||||
I find it quite easy to handle views different from my own. I feel more troubled when I see people agree with me for the wrong reasons.
|
I find it quite easy to handle views different from my own. I feel more troubled when I see people agree with me for the wrong reasons.
|
||||||
|
|
||||||
It's no secret that I'm a passionate supporter of software freedom: I've written two posts about how Free, Libre, and Open-Source software (<abbr title="Free, Libre, and Open-Source Software">FLOSS</abbr>) is necessary but insufficient to preserve user autonomy:
|
It's no secret that I'm a passionate supporter of software freedom: I've written two posts about how Free, Libre, and Open-Source software (<abbr title="Free, Libre, and Open-Source Software">FLOSS</abbr>) is necessary but insufficient to preserve user autonomy:
|
||||||
|
|
||||||
1. **[Whatsapp and the domestication of users](./../../../2021/01/27/whatsapp-and-the-domestication-of-users.html)**<br>The phenomenon of a class of predatory businesses models I call "user domestication" and defense measures: FLOSS, open platforms, and simplicity.
|
[Whatsapp and the domestication of users](./../../../2021/01/27/whatsapp-and-the-domestication-of-users.html "{itemprop='mentions'}")
|
||||||
2. **[Keeping platforms open](./../../../2021/02/23/keeping-platforms-open.html)**<br>How open platforms can lose their openness, and what measures can prevent this. The Web, XMPP, email, and Matrix are examples that highlight both sides of the issue.
|
: The phenomenon of a class of predatory businesses models I call "user domestication" and defense measures: FLOSS, open platforms, and simplicity.
|
||||||
|
|
||||||
|
[Keeping platforms open](./../../../2021/02/23/keeping-platforms-open.html "{itemprop='mentions'}")
|
||||||
|
: How open platforms can lose their openness, and what measures can prevent this. The Web, XMPP, email, and Matrix are examples that highlight both sides of the issue.
|
||||||
|
|
||||||
After two posts spanning over 5000 words, I need to add some nuance.
|
After two posts spanning over 5000 words, I need to add some nuance.
|
||||||
|
|
||||||
|
</section>
|
||||||
|
|
||||||
|
{{<toc>}}
|
||||||
|
|
||||||
|
<section role="doc-introduction">
|
||||||
|
|
||||||
|
Introduction
|
||||||
|
------------
|
||||||
|
|
||||||
One of the biggest parts of the Free and Open Source Software definitions is the freedom to study a program and modify it; in other words, access to editable source code. I agree that such access is essential; however, far too many people support source availability for the _wrong_ reasons. One such reason is that source code is necessary to have any degree of transparency into how a piece of software operates, and is therefore necessary to determine if it is at all secure or trustworthy. Although security through obscurity is certainly not a robust measure, this claim has two issues:
|
One of the biggest parts of the Free and Open Source Software definitions is the freedom to study a program and modify it; in other words, access to editable source code. I agree that such access is essential; however, far too many people support source availability for the _wrong_ reasons. One such reason is that source code is necessary to have any degree of transparency into how a piece of software operates, and is therefore necessary to determine if it is at all secure or trustworthy. Although security through obscurity is certainly not a robust measure, this claim has two issues:
|
||||||
|
|
||||||
- Source code describes what a program is designed to do; it is unnecessary and insufficient to determine if what it actually does aligns with its intended design.
|
- Source code describes what a program is designed to do; it is unnecessary and insufficient to determine if what it actually does aligns with its intended design.
|
||||||
|
|
@ -28,6 +45,8 @@ I'll update this post occasionally as I learn more on the subject. If you like i
|
||||||
|
|
||||||
_PS: this stance is not absolute; I concede to several good counter-arguments [at the bottom](#good-counter-arguments)!_
|
_PS: this stance is not absolute; I concede to several good counter-arguments [at the bottom](#good-counter-arguments)!_
|
||||||
|
|
||||||
|
</section>
|
||||||
|
|
||||||
How security fixes work
|
How security fixes work
|
||||||
-----------------------
|
-----------------------
|
||||||
|
|
||||||
|
|
@ -50,10 +69,15 @@ Understanding _how a program is designed_ is not the same as understanding _what
|
||||||
Source code[^2] is essential to describe a program's high-level, human-comprehensible design; it represents a contract that outlines how a developer _expects_ a program to behave. A compiler or interpreter[^3] must then translate it into machine instructions. But source code isn't always easy to map directly to machine instructions because it is part of a complex system:
|
Source code[^2] is essential to describe a program's high-level, human-comprehensible design; it represents a contract that outlines how a developer _expects_ a program to behave. A compiler or interpreter[^3] must then translate it into machine instructions. But source code isn't always easy to map directly to machine instructions because it is part of a complex system:
|
||||||
|
|
||||||
- Compilers (sometimes even interpreters) can apply optimizations and hardening measures that are difficult to reason about. This is especially true for <abbr title="Just-In-Time">JIT</abbr> compilers that leverage run-time information.
|
- Compilers (sometimes even interpreters) can apply optimizations and hardening measures that are difficult to reason about. This is especially true for <abbr title="Just-In-Time">JIT</abbr> compilers that leverage run-time information.
|
||||||
|
|
||||||
- The operating system itself may be poorly understood by the developers, and run a program in a way that contradicts a developer's expectations.
|
- The operating system itself may be poorly understood by the developers, and run a program in a way that contradicts a developer's expectations.
|
||||||
|
|
||||||
- Toolchains, interpreters, and operating systems can have bugs that impact program execution.
|
- Toolchains, interpreters, and operating systems can have bugs that impact program execution.
|
||||||
|
|
||||||
- Different compilers and compiler flags can offer different security guarantees and mitigations.
|
- Different compilers and compiler flags can offer different security guarantees and mitigations.
|
||||||
|
|
||||||
- Source code [can be deceptive](https://en.wikipedia.org/wiki/Underhanded_C_Contest) by featuring sneaky obfuscation techniques, sometimes unintentionally. Confusing naming patterns, re-definitions, and vulnerabilities masquerading as innocent bugs (plausible deniability; look up "hypocrite commits" for an example) have all been well-documented.
|
- Source code [can be deceptive](https://en.wikipedia.org/wiki/Underhanded_C_Contest) by featuring sneaky obfuscation techniques, sometimes unintentionally. Confusing naming patterns, re-definitions, and vulnerabilities masquerading as innocent bugs (plausible deniability; look up "hypocrite commits" for an example) have all been well-documented.
|
||||||
|
|
||||||
- All of the above points apply to each dependency and the underlying operating system, which can impact a program's behavior.
|
- All of the above points apply to each dependency and the underlying operating system, which can impact a program's behavior.
|
||||||
|
|
||||||
Furthermore, all programmers are flawed mortals who don't always fully understand source code. Everyone who's done a non-trivial amount of programming is familiar with the feeling of encountering a bug during run-time for which the cause is impossible to find...until they notice it staring them in the face on Line 12. Think of all the bugs that _aren't_ so easily noticed.
|
Furthermore, all programmers are flawed mortals who don't always fully understand source code. Everyone who's done a non-trivial amount of programming is familiar with the feeling of encountering a bug during run-time for which the cause is impossible to find...until they notice it staring them in the face on Line 12. Think of all the bugs that _aren't_ so easily noticed.
|
||||||
|
|
@ -126,7 +150,7 @@ Unfortunately, some components are poorly understood due to being obfuscated usi
|
||||||
|
|
||||||
Skochinsky's and Corna's analysis was sufficient to clarify (but not completely contradict) sensationalism claiming that ME can remotely lock any PC (it was a former opt-in feature), can spy on anything the user does (they clarified that access is limited to unblocked parts of the host memory and the integrated GPU, but doesn't include e.g. the framebuffer), etc.
|
Skochinsky's and Corna's analysis was sufficient to clarify (but not completely contradict) sensationalism claiming that ME can remotely lock any PC (it was a former opt-in feature), can spy on anything the user does (they clarified that access is limited to unblocked parts of the host memory and the integrated GPU, but doesn't include e.g. the framebuffer), etc.
|
||||||
|
|
||||||
While claims such as "ME is a black box that can do anything" are misleading, ME not without its share of vulnerabilities. My favorite look at its issues is a presentation by {{<indieweb-person first-name="Mark" last-name="Ermolov" url="https://www.blackhat.com/eu-17/speakers/Mark-Ermolov.html">}} and {{<indieweb-person first-name="Maxim" last-name="Goryachy" url="https://www.blackhat.com/eu-17/speakers/Maxim-Goryachy.html">}} at Black Hat Europe 2017: [How to Hack a Turned-Off Computer, or Running Unsigned Code in Intel Management Engine](https://www.blackhat.com/docs/eu-17/materials/eu-17-Goryachy-How-To-Hack-A-Turned-Off-Computer-Or-Running-Unsigned-Code-In-Intel-Management-Engine-wp.pdf).
|
While claims such as "ME is a black box that can do anything" are misleading, ME not without its share of vulnerabilities. My favorite look at its issues is a presentation by <span itemprop="mentions" itemscope itemtype="https://schema.org/PresentationDigitalDocument">{{<indieweb-person itemprop="author" first-name="Mark" last-name="Ermolov" url="https://www.blackhat.com/eu-17/speakers/Mark-Ermolov.html">}} and {{<indieweb-person itemprop="author" first-name="Maxim" last-name="Goryachy" url="https://www.blackhat.com/eu-17/speakers/Maxim-Goryachy.html">}} at Black Hat Europe 2017: [How to Hack a Turned-Off Computer, or Running Unsigned Code in Intel Management Engine](https://www.blackhat.com/docs/eu-17/materials/eu-17-Goryachy-How-To-Hack-A-Turned-Off-Computer-Or-Running-Unsigned-Code-In-Intel-Management-Engine-wp.pdf)</span>.
|
||||||
|
|
||||||
In short: ME being proprietary doesn't mean that we can't find out how (in)secure it is. Binary analysis when paired with runtime inspection can give us a good understanding of what trade-offs we make by using it. While ME has a history of serious vulnerabilities, they're nowhere near what [borderline conspiracy theories](https://web.archive.org/web/20210302072839/themerkle.com/what-is-the-intel-management-engine-backdoor/) claim.[^11]
|
In short: ME being proprietary doesn't mean that we can't find out how (in)secure it is. Binary analysis when paired with runtime inspection can give us a good understanding of what trade-offs we make by using it. While ME has a history of serious vulnerabilities, they're nowhere near what [borderline conspiracy theories](https://web.archive.org/web/20210302072839/themerkle.com/what-is-the-intel-management-engine-backdoor/) claim.[^11]
|
||||||
|
|
||||||
|
|
@ -141,14 +165,17 @@ Fuzzing doesn't necessarily depend on access to source code, as it is a black-bo
|
||||||
|
|
||||||
Fuzzing frequently catches bugs that are only apparent by running a program, not by reading source code. Even so, the biggest beneficiaries of fuzzing are open source projects. [cURL](https://github.com/curl/curl-fuzzer), [OpenSSL](https://github.com/openssl/openssl/tree/master/fuzz), web browsers, text rendering libraries (HarfBuzz, FreeType) and toolchains (GCC, Clang, the official Go toolchain, etc.) are some notable examples.
|
Fuzzing frequently catches bugs that are only apparent by running a program, not by reading source code. Even so, the biggest beneficiaries of fuzzing are open source projects. [cURL](https://github.com/curl/curl-fuzzer), [OpenSSL](https://github.com/openssl/openssl/tree/master/fuzz), web browsers, text rendering libraries (HarfBuzz, FreeType) and toolchains (GCC, Clang, the official Go toolchain, etc.) are some notable examples.
|
||||||
|
|
||||||
<figure itemscope itemtype="https://schema.org/Quotation">
|
{{<quotation>}}
|
||||||
<blockquote itemprop="text">
|
<blockquote itemprop="text">
|
||||||
<p>I've said it before but let me say it again: fuzzing is really the top method to find problems in curl once we've fixed all flaws that the static analyzers we use have pointed out. The primary fuzzing for curl is done by OSS-Fuzz, that tirelessly keeps hammering on the most recent curl code.</p>
|
|
||||||
</blockquote>
|
I've said it before but let me say it again: fuzzing is really the top method to find problems in curl once we've fixed all flaws that the static analyzers we use have pointed out. The primary fuzzing for curl is done by OSS-Fuzz, that tirelessly keeps hammering on the most recent curl code.
|
||||||
<figcaption class="h-cite" itemprop="citation">
|
|
||||||
— {{<indieweb-person first-name="Daniel" last-name="Stenberg" url="https://daniel.haxx.se/" itemprop="author">}}, <cite itemprop="isPartOf" itemscope itemtype="https://schema.org/BlogPosting"><a class="u-url p-name" itemprop="url" href="https://daniel.haxx.se/blog/2020/09/23/a-google-grant-for-libcurl-work/"><span itemprop="name">A Google grant for libcurl work</span></a></cite>
|
</blockquote>
|
||||||
</figcaption>
|
{{<quotecaption partOfType="BlogPosting">}}
|
||||||
</figure>
|
{{<indieweb-person first-name="Daniel" last-name="Stenberg" url="https://daniel.haxx.se/" itemprop="author">}},
|
||||||
|
{{<cited-work url="https://daniel.haxx.se/blog/2020/09/23/a-google-grant-for-libcurl-work/" name="A Google grant for libcurl work" extraName="headline">}}
|
||||||
|
{{</quotecaption>}}
|
||||||
|
{{</quotation>}}
|
||||||
|
|
||||||
If you want to get started with fuzzing, I recommend checking out [the quick-start guide for American Fuzzy Loop](https://github.com/google/AFL/blob/master/docs/QuickStartGuide.txt). Some languages like Go 1.18 also have fuzzing tools available right in the standard library.
|
If you want to get started with fuzzing, I recommend checking out [the quick-start guide for American Fuzzy Loop](https://github.com/google/AFL/blob/master/docs/QuickStartGuide.txt). Some languages like Go 1.18 also have fuzzing tools available right in the standard library.
|
||||||
|
|
||||||
|
|
@ -170,25 +197,31 @@ Good counter-arguments
|
||||||
I readily concede to several points in favor of source availability from a security perspective:
|
I readily concede to several points in favor of source availability from a security perspective:
|
||||||
|
|
||||||
- Source code can make analysis _easier_ by _supplementing_ source-independent approaches. The lines between the steps I mentioned in the [four-step vulnerability-fixing process](#how-security-fixes-work) are blurry.
|
- Source code can make analysis _easier_ by _supplementing_ source-independent approaches. The lines between the steps I mentioned in the [four-step vulnerability-fixing process](#how-security-fixes-work) are blurry.
|
||||||
|
|
||||||
- Patching vulnerabilities is important. Source availability makes it possible for the community, package maintainers, or reporters of a vulnerability to patch software. Package maintainers often blur the line between "packager" and "contributor" by helping projects migrate away from abandoned/insecure dependencies. One example that comes to mind is the Python 2 to Python 3 transition for projects like Calibre.[^12] Being able to fix issues independent of upstream support is an important mitigation against [user domestication](./../../../2021/01/27/whatsapp-and-the-domestication-of-users.html).
|
- Patching vulnerabilities is important. Source availability makes it possible for the community, package maintainers, or reporters of a vulnerability to patch software. Package maintainers often blur the line between "packager" and "contributor" by helping projects migrate away from abandoned/insecure dependencies. One example that comes to mind is the Python 2 to Python 3 transition for projects like Calibre.[^12] Being able to fix issues independent of upstream support is an important mitigation against [user domestication](./../../../2021/01/27/whatsapp-and-the-domestication-of-users.html).
|
||||||
|
|
||||||
- Some developers/vendors don't distribute binaries that make use of modern toolchain-level exploit mitigations (e.g. <abbr title="Position-Independent Executables">PIE</abbr>, <abbr title="ReLocation Read-Only">RELRO</abbr>, stack canaries, automatic variable initialization, [<abbr title="Control-Flow Integrity">CFI</abbr>](https://clang.llvm.org/docs/ControlFlowIntegrity.html), etc.[^13]). In these cases, building software yourself with these mitigations (or delegating it to a distro that enforces them) requires source code availability (or at least some sort of intermediate representation).
|
- Some developers/vendors don't distribute binaries that make use of modern toolchain-level exploit mitigations (e.g. <abbr title="Position-Independent Executables">PIE</abbr>, <abbr title="ReLocation Read-Only">RELRO</abbr>, stack canaries, automatic variable initialization, [<abbr title="Control-Flow Integrity">CFI</abbr>](https://clang.llvm.org/docs/ControlFlowIntegrity.html), etc.[^13]). In these cases, building software yourself with these mitigations (or delegating it to a distro that enforces them) requires source code availability (or at least some sort of intermediate representation).
|
||||||
|
|
||||||
- Closed-source software may or may not have builds available that include sanitizers and debug symbols.
|
- Closed-source software may or may not have builds available that include sanitizers and debug symbols.
|
||||||
|
|
||||||
- Although fuzzing release binaries is possible, fuzzing is much easier to do when source code is available. Vendors of proprietary software seldom release special fuzz-friendly builds, and filtering out false-positives can be quite tedious without understanding high-level design.
|
- Although fuzzing release binaries is possible, fuzzing is much easier to do when source code is available. Vendors of proprietary software seldom release special fuzz-friendly builds, and filtering out false-positives can be quite tedious without understanding high-level design.
|
||||||
- It is certainly possible to notice a vulnerability in source code. Excluding low-hanging fruit typically caught by static code analysis and peer review, it's not the main way most vulnerabilities are found nowadays (thanks to {{<indieweb-person nickname="X_CLI" url="https://www.broken-by-design.fr/">}} for [reminding me about what source analysis does accomplish](https://lemmy.ml/post/167321/comment/117774)).
|
|
||||||
|
- It is certainly possible to notice a vulnerability in source code. Excluding low-hanging fruit typically caught by static code analysis and peer review, it's not the main way most vulnerabilities are found nowadays (thanks to {{<indieweb-person itemprop="mentions" nickname="X_CLI" url="https://www.broken-by-design.fr/">}} for [reminding me about what source analysis does accomplish](https://lemmy.ml/post/167321/comment/117774)).
|
||||||
|
|
||||||
- Software as a Service can be incredibly difficult to analyze, as we typically have little more than the ability to query a server. Servers don't send core dumps, server-side binaries, or trace logs for analysis. Further­more, it's difficult to verify which software a server is running.[^14] For services that require trusting a server, access to the server-side software is important from both a security and a user-freedom perspective
|
- Software as a Service can be incredibly difficult to analyze, as we typically have little more than the ability to query a server. Servers don't send core dumps, server-side binaries, or trace logs for analysis. Further­more, it's difficult to verify which software a server is running.[^14] For services that require trusting a server, access to the server-side software is important from both a security and a user-freedom perspective
|
||||||
|
|
||||||
Most of this post is written with the assumption that binaries are inspectable and traceable. Binary obfuscation and some forms of content protection/<abbr title="Digital Rights Management">DRM</abbr> violate this assumption and actually do make analysis more difficult.
|
Most of this post is written with the assumption that binaries are inspectable and traceable. Binary obfuscation and some forms of content protection/<abbr title="Digital Rights Management">DRM</abbr> violate this assumption and actually do make analysis more difficult.
|
||||||
|
|
||||||
Beyond source code, transparency into the development helps assure users of compliance with good security practices. Viewing VCS history, patch reviews, linter configurations, etc. reveal the standards that code is being held up to, some of which can be related to bug-squashing and security.
|
Beyond source code, transparency into the development helps assure users of compliance with good security practices. Viewing VCS history, patch reviews, linter configurations, etc. reveal the standards that code is being held up to, some of which can be related to bug-squashing and security.
|
||||||
|
|
||||||
{{<indieweb-person nickname="Patience" url="https://matrix.to/#/@hypokeimenon:tchncs.de">}} on Matrix also had a great response, which I agree with and adapt below:
|
{{<indieweb-person itemprop="mentions" nickname="Patience" url="https://matrix.to/#/@hypokeimenon:tchncs.de">}} on Matrix also had a great response, which I agree with and adapt below:
|
||||||
|
|
||||||
Whether or not the source code is available for software does not change how insecure it is. However, there are good security-related incentives to publish source code.
|
Whether or not the source code is available for software does not change how insecure it is. However, there are good security-related incentives to publish source code.
|
||||||
|
|
||||||
- Doing so improves vulnerability patchability and future architectural improvement by lowering the barrier to contribution. The fixes that follow can be _shared and used by other projects_ across the field, some of which can in turn be used by the vendor. This isn't a zero-sum game; a rising tide lifts all boats.
|
- Doing so improves vulnerability patchability and future architectural improvement by lowering the barrier to contribution. The fixes that follow can be _shared and used by other projects_ across the field, some of which can in turn be used by the vendor. This isn't a zero-sum game; a rising tide lifts all boats.
|
||||||
- It's generally good practice to assume an attacker has full knowledge of a system instead of relying on security through obscurity. Releasing code provides strong assurance that this assumption is being made. It's a way for vendors to put their money where their mouth is.
|
- It's generally good practice to assume an attacker has full knowledge of a system instead of relying on security through obscurity. Releasing code provides strong assurance that this assumption is being made. It's a way for vendors to put their money where their mouth is.
|
||||||
|
|
||||||
Both Patience and {{<indieweb-person first-name="Drew" last-name="Devault" url="https://drewdevault.com/">}} argue that given the above points, a project whose goal is maximum security would release code. Strictly speaking, I agree. Good intentions don't imply good results, but they can _supplement_ good results to provide some trust in a project's future.
|
Both Patience and {{<indieweb-person itemprop="mentions" first-name="Drew" last-name="Devault" url="https://drewdevault.com/">}} argue that given the above points, a project whose goal is maximum security would release code. Strictly speaking, I agree. Good intentions don't imply good results, but they can _supplement_ good results to provide some trust in a project's future.
|
||||||
|
|
||||||
Con­clusion {#conclusion}
|
Con­clusion {#conclusion}
|
||||||
---------------
|
---------------
|
||||||
|
|
@ -208,7 +241,7 @@ Releasing source code is just one thing vendors can do to improve audits; other
|
||||||
|
|
||||||
[^3]: Or a JIT compiler, or a [bunch of clockwork](https://en.wikipedia.org/wiki/Analytical_Engine), or...
|
[^3]: Or a JIT compiler, or a [bunch of clockwork](https://en.wikipedia.org/wiki/Analytical_Engine), or...
|
||||||
|
|
||||||
[^4]: For completeness, I should add that there is one source-based approach that can verify correctness: formal proofs. Functional programming languages that [support dependent types](https://en.wikipedia.org/wiki/Dependent_type) can be provably correct at the source level. Assuming their self-hosted toolchains have similar guarantees, developers using these languages might have to worry less about bugs they couldn't find in the source code. This can alleviate concerns that their language runtimes can make it hard to reason about low-level behavior. Thanks to {{<indieweb-person first-name="Adrian" last-name="Cochrane" url="https://adrian.geek.nz/">}} for pointing this out.
|
[^4]: For completeness, I should add that there is one source-based approach that can verify correctness: formal proofs. Functional programming languages that [support dependent types](https://en.wikipedia.org/wiki/Dependent_type) can be provably correct at the source level. Assuming their self-hosted toolchains have similar guarantees, developers using these languages might have to worry less about bugs they couldn't find in the source code. This can alleviate concerns that their language runtimes can make it hard to reason about low-level behavior. Thanks to {{<indieweb-person itemprop="mentions" first-name="Adrian" last-name="Cochrane" url="https://adrian.geek.nz/">}} for pointing this out.
|
||||||
|
|
||||||
[^5]: For example: C, C++, Objective-C, Go, Fortran, and others can utilize sanitizers from Clang and/or GCC.
|
[^5]: For example: C, C++, Objective-C, Go, Fortran, and others can utilize sanitizers from Clang and/or GCC.
|
||||||
|
|
||||||
|
|
@ -224,7 +257,7 @@ Releasing source code is just one thing vendors can do to improve audits; other
|
||||||
|
|
||||||
[^11]: As an aside: your security isn't necessarily improved by "disabling" it, since it still runs during the initial boot sequence and does provide some hardening measures of its own (e.g., a <abbr title="Trusted Platform Module">TPM</abbr>).
|
[^11]: As an aside: your security isn't necessarily improved by "disabling" it, since it still runs during the initial boot sequence and does provide some hardening measures of its own (e.g., a <abbr title="Trusted Platform Module">TPM</abbr>).
|
||||||
|
|
||||||
[^12]: In 2017, Calibre's author actually wanted to stay with Python 2 after its EOL date, and [maintain Python 2 himself](https://bugs.launchpad.net/calibre/+bug/1714107). Users and package maintainers were quite unhappy with this, as Python 2 would no longer be receiving security fixes after 2020. While official releases of Calibre use a bundled Python interpreter, distro packages typically use the system Python package; Calibre's popularity and insistence on using Python 2 made it a roadblock to getting rid of the Python 2 package in most distros. What eventually happened was that community members (especially {{<indieweb-person first-name="Eli" last-name="Schwartz" url="https://github.com/eli-schwartz">}} and {{<indieweb-person first-name="Flaviu" last-name="Tamas" url="https://flaviutamas.com/">}}) submitted patches to migrate Calibre away from Python 2. Calibre migrated to Python 3 by [version 5.0](https://calibre-ebook.com/new-in/fourteen).
|
[^12]: In 2017, Calibre's author actually wanted to stay with Python 2 after its EOL date, and [maintain Python 2 himself](https://bugs.launchpad.net/calibre/+bug/1714107). Users and package maintainers were quite unhappy with this, as Python 2 would no longer be receiving security fixes after 2020. While official releases of Calibre use a bundled Python interpreter, distro packages typically use the system Python package; Calibre's popularity and insistence on using Python 2 made it a roadblock to getting rid of the Python 2 package in most distros. What eventually happened was that community members (especially {{<indieweb-person itemprop="mentions" first-name="Eli" last-name="Schwartz" url="https://github.com/eli-schwartz">}} and {{<indieweb-person itemprop="mentions" first-name="Flaviu" last-name="Tamas" url="https://flaviutamas.com/">}}) submitted patches to migrate Calibre away from Python 2. Calibre migrated to Python 3 by [version 5.0](https://calibre-ebook.com/new-in/fourteen).
|
||||||
|
|
||||||
[^13]: Linux distributions' CFI+<abbr title="Adress-Space Layout Randomization">ASLR</abbr> implementations rely executables compiled with CFI+PIE support, and ideally with stack-smashing protectors and no-execute bits. These implementations are flawed (see [On the Effectiveness of Full-ASLR on 64-bit Linux](https://web.archive.org/web/20211021222659/http://cybersecurity.upv.es/attacks/offset2lib/offset2lib-paper.pdf) and [Brad Spengler's presentation comparing these with PaX's own implementation](https://grsecurity.net/PaX-presentation.pdf)).
|
[^13]: Linux distributions' CFI+<abbr title="Adress-Space Layout Randomization">ASLR</abbr> implementations rely executables compiled with CFI+PIE support, and ideally with stack-smashing protectors and no-execute bits. These implementations are flawed (see [On the Effectiveness of Full-ASLR on 64-bit Linux](https://web.archive.org/web/20211021222659/http://cybersecurity.upv.es/attacks/offset2lib/offset2lib-paper.pdf) and [Brad Spengler's presentation comparing these with PaX's own implementation](https://grsecurity.net/PaX-presentation.pdf)).
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,25 +1,33 @@
|
||||||
---
|
---
|
||||||
date: "2020-11-17T13:13:03-08:00"
|
date: "2020-11-17T13:13:03-08:00"
|
||||||
description: A seires on setting up resilient git-based project workflows, free of
|
description: A seires on setting up resilient git-based project workflows, free of vendor lock-in.
|
||||||
vendor lock-in.
|
|
||||||
outputs:
|
outputs:
|
||||||
- html
|
- html
|
||||||
- gemtext
|
- gemtext
|
||||||
tags:
|
tags:
|
||||||
- git
|
- git
|
||||||
- foss
|
- foss
|
||||||
title: "Resilient Git, Part 0: Introduction"
|
title: "Resilient Git, Part 0: Introduction"
|
||||||
---
|
---
|
||||||
|
|
||||||
|
<div itemprop="backstory">
|
||||||
|
|
||||||
Recently, GitHub re-instated the [youtube-dl git repository](https://github.com/ytdl-org/youtube-dl) after following a takedown request by the RIAA under the DMCA. Shortly after the takedown, many members of the community showed great interest in "decentralizing git" and setting up a more resilient forge. What many of these people fail to understand is that the Git-based project setup is designed to support decentralization by being fully distributed.
|
Recently, GitHub re-instated the [youtube-dl git repository](https://github.com/ytdl-org/youtube-dl) after following a takedown request by the RIAA under the DMCA. Shortly after the takedown, many members of the community showed great interest in "decentralizing git" and setting up a more resilient forge. What many of these people fail to understand is that the Git-based project setup is designed to support decentralization by being fully distributed.
|
||||||
|
|
||||||
Following the drama, I'm putting together a multi-part guide on how to leverage the decentralized, distributed nature of git and its ecosystem. I made every effort to include all parts of a typical project.
|
Following the drama, I'm putting together a multi-part guide on how to leverage the decentralized, distributed nature of git and its ecosystem. I made every effort to include all parts of a typical project.
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
I'll update this post as I add articles to the series. At the moment, I've planned to write the following articles:
|
I'll update this post as I add articles to the series. At the moment, I've planned to write the following articles:
|
||||||
|
|
||||||
1. [Hydra Hosting](/2020/11/18/git-workflow-1.html): repository hosting.
|
1. [Hydra Hosting](/2020/11/18/git-workflow-1.html): repository hosting.
|
||||||
|
|
||||||
2. Community feedback (issues, support, etc.)
|
2. Community feedback (issues, support, etc.)
|
||||||
|
|
||||||
3. Community contributions (patches)
|
3. Community contributions (patches)
|
||||||
|
|
||||||
4. CI/CD
|
4. CI/CD
|
||||||
|
|
||||||
5. Distribution
|
5. Distribution
|
||||||
|
|
||||||
The result of the workflows this series covers will be minimal dependence on outside parties; all members of the community will easily be able to get a copy of the software, its code, development history, issues, and patches offline on their machines with implementation-neutral open standards. Following open standards is the killer feature: nothing in this workflow depends on a specific platform (GitHub, GitLab, Gitea, Bitbucket, Docker, Nix, Jenkins, etc.), almost eliminating your project's "bus factor".
|
The result of the workflows this series covers will be minimal dependence on outside parties; all members of the community will easily be able to get a copy of the software, its code, development history, issues, and patches offline on their machines with implementation-neutral open standards. Following open standards is the killer feature: nothing in this workflow depends on a specific platform (GitHub, GitLab, Gitea, Bitbucket, Docker, Nix, Jenkins, etc.), almost eliminating your project's "bus factor".
|
||||||
|
|
|
||||||
|
|
@ -2,14 +2,23 @@
|
||||||
date: "2020-11-18T18:31:15-08:00"
|
date: "2020-11-18T18:31:15-08:00"
|
||||||
description: Efficient redundancy via repository mirroring with nothing but git.
|
description: Efficient redundancy via repository mirroring with nothing but git.
|
||||||
outputs:
|
outputs:
|
||||||
- html
|
- html
|
||||||
- gemtext
|
- gemtext
|
||||||
tags:
|
tags:
|
||||||
- git
|
- git
|
||||||
- foss
|
- foss
|
||||||
title: "Resilient Git, Part 1: Hydra Hosting"
|
title: "Resilient Git, Part 1: Hydra Hosting"
|
||||||
---
|
---
|
||||||
This is Part 1 of a series called [Resilient Git](/2020/11/17/git-workflow-0.html).
|
<div role="note">
|
||||||
|
|
||||||
|
This is Part 1 of a series called [Resilient Git](/2020/11/17/git-workflow-0.html "{itemprop='relatedLink'}").
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<section role="doc-introduction">
|
||||||
|
|
||||||
|
Intro­duction {#introduction}
|
||||||
|
-----------------
|
||||||
|
|
||||||
The most important part of a project is its code. Resilient projects should have their code in multiple places of equal weight so that work continues normally if a single remote goes down.
|
The most important part of a project is its code. Resilient projects should have their code in multiple places of equal weight so that work continues normally if a single remote goes down.
|
||||||
|
|
||||||
|
|
@ -19,6 +28,8 @@ Having multiple primary remotes of equal status might sound like a bad idea. If
|
||||||
|
|
||||||
No. Of course not. A good distributed system should automatically keep its nodes in sync to avoid the hassle of checking multiple places for updates.
|
No. Of course not. A good distributed system should automatically keep its nodes in sync to avoid the hassle of checking multiple places for updates.
|
||||||
|
|
||||||
|
</section>
|
||||||
|
|
||||||
Adding remotes
|
Adding remotes
|
||||||
--------------
|
--------------
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,6 @@
|
||||||
---
|
---
|
||||||
date: "2021-02-23T11:54:00-08:00"
|
date: "2021-02-23T11:54:00-08:00"
|
||||||
|
lastmod: "2021-03-02T23:04:04-08:00"
|
||||||
description: "How open platforms become closed, and how standards-driven development can prevent it from happening."
|
description: "How open platforms become closed, and how standards-driven development can prevent it from happening."
|
||||||
footnote_heading: "Notes"
|
footnote_heading: "Notes"
|
||||||
outputs:
|
outputs:
|
||||||
|
|
@ -13,12 +14,19 @@ title: Keeping platforms open
|
||||||
---
|
---
|
||||||
This article is the second entry of series of posts exploring situations in which <abbr title="free, libre, and open-source software">FLOSS</abbr> alone isn't enough to secure user freedom.
|
This article is the second entry of series of posts exploring situations in which <abbr title="free, libre, and open-source software">FLOSS</abbr> alone isn't enough to secure user freedom.
|
||||||
|
|
||||||
|
<section role="doc-introduction">
|
||||||
|
|
||||||
|
Intro­duction {#introduction}
|
||||||
|
-----------------
|
||||||
|
|
||||||
My previous article, [Whatsapp and the domestication of users](/2021/01/27/whatsapp-and-the-domestication-of-users.html), got more attention than I was expecting. Some responses gave me a lot to think about,[^1] especially regarding _actions_ we can take. I suggest reading that article first; it explained what "user domestication" is and why it's a problem. It enumerated three countermeasures: FLOSS, simplicity, and open platforms.
|
My previous article, [Whatsapp and the domestication of users](/2021/01/27/whatsapp-and-the-domestication-of-users.html), got more attention than I was expecting. Some responses gave me a lot to think about,[^1] especially regarding _actions_ we can take. I suggest reading that article first; it explained what "user domestication" is and why it's a problem. It enumerated three countermeasures: FLOSS, simplicity, and open platforms.
|
||||||
|
|
||||||
Hard problems, by definition, lack easy solutions. Simply choosing (or creating) a platform that avoids user domestication isn't enough if that platform can change. The price of freedom is eternal vigilance; in addition to settling on the right platform, we must ensure that it honors its users in both the present _and the future_. Keeping a platform FLOSS and simple is more straightforward[^2] than keeping a platform "open".
|
Hard problems, by definition, lack easy solutions. Simply choosing (or creating) a platform that avoids user domestication isn't enough if that platform can change. The price of freedom is eternal vigilance; in addition to settling on the right platform, we must ensure that it honors its users in both the present _and the future_. Keeping a platform FLOSS and simple is more straightforward[^2] than keeping a platform "open".
|
||||||
|
|
||||||
How do we keep an open platform from becoming a closed platform in the future?
|
How do we keep an open platform from becoming a closed platform in the future?
|
||||||
|
|
||||||
|
</section>
|
||||||
|
|
||||||
How open platforms become closed
|
How open platforms become closed
|
||||||
--------------------------------
|
--------------------------------
|
||||||
|
|
||||||
|
|
@ -70,9 +78,22 @@ Platforms are more than their protocols. Different implementations have unique b
|
||||||
|
|
||||||
After reading my previous article, a few people contacted me to ask for my thoughts regarding certain email providers. There's not much that can set a standard email provider apart if it just hosts a simple email server. To distinguish themselves, email providers often implement many features beyond email standards compliance.
|
After reading my previous article, a few people contacted me to ask for my thoughts regarding certain email providers. There's not much that can set a standard email provider apart if it just hosts a simple email server. To distinguish themselves, email providers often implement many features beyond email standards compliance.
|
||||||
|
|
||||||
The vast majority of email accounts come from a small handful of dominant providers backed by large companies (Gmail, Yahoo! Mail, Yandex Mail, Mail.ru, iCloud, and others). Providers such as Gmail are notorious for implementing advanced spam filters prejudiced against non-mainstream email providers. Users who self-host email servers or use small providers frequently trigger false positives and end up having their messages incorrectly labeled as spam until they can build up a "reputation".[^4] The addition of such a complex spam-prevention filter strengthens the email oligopoly by creating a barrier to entry for newcomers. Low-volume senders are discriminated against, as Migadu [found out](https://archive.is/rJnSs#deliverability):
|
The vast majority of email accounts come from a small handful of dominant providers backed by large companies (Gmail, Yahoo! Mail, Yandex Mail, Mail.ru, iCloud, and others). Providers such as Gmail are notorious for implementing advanced spam filters prejudiced against non-mainstream email providers. Users who self-host email servers or use small providers frequently trigger false positives and end up having their messages incorrectly labeled as spam until they can build up a "reputation".[^4] The addition of such a complex spam-prevention filter strengthens the email oligopoly by creating a barrier to entry for newcomers. Low-volume senders are discriminated against, as Migadu found out:
|
||||||
|
|
||||||
> We’ve already seen our share of bad spam filters and misconfigured servers. In some cases recipient servers intentionally rejected correct emails just because we are a low volume sender. Ironically that is how an ideal sender should be. To improve the “receiveability” they of course offer their own hosted email service at a hefty price.
|
{{<quotation>}}
|
||||||
|
|
||||||
|
<blockquote itemprop="text">
|
||||||
|
|
||||||
|
We've already seen our share of bad spam filters and misconfigured servers. In some cases recipient servers intentionally rejected correct emails just because we are a low volume sender. Ironically that is how an ideal sender should be. To improve the "receiveability" they of course offer their own hosted email service at a hefty price.
|
||||||
|
|
||||||
|
</blockquote>
|
||||||
|
{{<quotecaption partOfType="Article">}}
|
||||||
|
|
||||||
|
{{<cited-work name="How's Migadu's Deliverability?" url="https://archive.is/rJnSs#deliverability" extraName="headline">}}, section "So, Finally, How’s The Deliverability?"
|
||||||
|
|
||||||
|
{{</quotecaption>}}
|
||||||
|
|
||||||
|
{{</quotation>}}
|
||||||
|
|
||||||
Another example: email providers such as Hey.com, Protonmail, and Tutanota offer many features that are incompatible with IMAP/POP3. Protonmail and Tutanota use their own non-standard E2EE implementation (rather than focusing on improving the UX for vanilla PGP), and Hey.com offers server-side mail organization. Users of these services must use official Web, desktop, and mobile clients.[^5] These three providers control both the client and the server, giving them the means for vendor lock-in. Of course, there's a limit to the amount of lock-in these providers can achieve: as I explained in the [XMPP case study](#case-study-the-boxing-of-xmpp), these providers still need to support SMTP to stay compatible with the wider email landscape.
|
Another example: email providers such as Hey.com, Protonmail, and Tutanota offer many features that are incompatible with IMAP/POP3. Protonmail and Tutanota use their own non-standard E2EE implementation (rather than focusing on improving the UX for vanilla PGP), and Hey.com offers server-side mail organization. Users of these services must use official Web, desktop, and mobile clients.[^5] These three providers control both the client and the server, giving them the means for vendor lock-in. Of course, there's a limit to the amount of lock-in these providers can achieve: as I explained in the [XMPP case study](#case-study-the-boxing-of-xmpp), these providers still need to support SMTP to stay compatible with the wider email landscape.
|
||||||
|
|
||||||
|
|
@ -108,7 +129,7 @@ For example, Element and the Matrix.org Foundation would alleviate most of my co
|
||||||
Drawbacks
|
Drawbacks
|
||||||
---------
|
---------
|
||||||
|
|
||||||
The biggest drawback to the advice I've presented is development speed. Keeping compatibility and spec compliance slows down the rate at which new features can be added. As Moxie [argues](https://signal.org/blog/the-ecosystem-is-moving/), Signal might not have been able to implement as many features if it was an open platform; spec-constrained development is, by definition, _constrained_. Users are limited by the lowest common denominator among popular participating implementations.
|
The biggest drawback to the advice I've presented is development speed. Keeping compatibility and spec compliance slows down the rate at which new features can be added. [As Moxie argues](https://signal.org/blog/the-ecosystem-is-moving/), Signal might not have been able to implement as many features if it was an open platform; spec-constrained development is, by definition, _constrained_. Users are limited by the lowest common denominator among popular participating implementations.
|
||||||
|
|
||||||
Open platforms with multiple providers and implementations often suffer from poorer usability, especially with regards to onboarding. Instead of just opening the official app/website, users need to choose from multiple clients and providers. This can be a turn-off for casual users just wanting to try something out. One of the best ways to improve the onboarding experience is to offer recommendations to your non-technical friends; you know them well and can probably help them make an informed decision.
|
Open platforms with multiple providers and implementations often suffer from poorer usability, especially with regards to onboarding. Instead of just opening the official app/website, users need to choose from multiple clients and providers. This can be a turn-off for casual users just wanting to try something out. One of the best ways to improve the onboarding experience is to offer recommendations to your non-technical friends; you know them well and can probably help them make an informed decision.
|
||||||
|
|
||||||
|
|
@ -117,18 +138,29 @@ Parallels to other situations
|
||||||
|
|
||||||
Programming languages driven by a standard rather than a reference implementation typically have greater portability, many good implementations, and are unlikely to fade away with time. Examples include C, C++, Common Lisp, JavaScript, and POSIX Shell. Compare this with a language like Python: so many packages depend on the CPython reference implementation's approach to C extensions that alternative implementations such as PyPy must perpetually remain second-class citizens.
|
Programming languages driven by a standard rather than a reference implementation typically have greater portability, many good implementations, and are unlikely to fade away with time. Examples include C, C++, Common Lisp, JavaScript, and POSIX Shell. Compare this with a language like Python: so many packages depend on the CPython reference implementation's approach to C extensions that alternative implementations such as PyPy must perpetually remain second-class citizens.
|
||||||
|
|
||||||
The standards- and consensus-driven approach to platform development and the inefficiency that comes with it is a trade-off visible in many places outside software development. Most forms of democracy suffer from bureaucracy and in-fighting that stifle progress. Some have argued that democracy's inefficiency is a feature, not a bug. As Nathan Myhrvold [puts it](https://slate.com/news-and-politics/1996/10/the-virtue-of-inefficient-government.html):
|
The standards- and consensus-driven approach to platform development and the inefficiency that comes with it is a trade-off visible in many places outside software development. Most forms of democracy suffer from bureaucracy and in-fighting that stifle progress. Some have argued that democracy's inefficiency is a feature, not a bug. As Nathan Myhrvold puts it:
|
||||||
|
|
||||||
> The reason societies with democratic governments are better places to live in than their alternatives isn’t because of some goodness intrinsic to democracy, but because its hopeless inefficiency helps blunt the basic potential for evil. The constraint of maintaining constant popularity is simply too large a burden to bear. So, happily, very little gets done that is extremely bad–or extremely good.
|
{{<quotation>}}
|
||||||
|
|
||||||
|
<blockquote itemprop="text">
|
||||||
|
|
||||||
|
The reason societies with democratic governments are better places to live in than their alternatives isn’t because of some goodness intrinsic to democracy, but because its hopeless inefficiency helps blunt the basic potential for evil. The constraint of maintaining constant popularity is simply too large a burden to bear. So, happily, very little gets done that is extremely bad–or extremely good.
|
||||||
|
|
||||||
Perhaps the biggest benefit to abandoning the "move fast and break things" mindset is that in addition to making it hard to rapidly improve a service, abandoning the mindset also makes it hard to rapidly worsen a service.
|
Perhaps the biggest benefit to abandoning the "move fast and break things" mindset is that in addition to making it hard to rapidly improve a service, abandoning the mindset also makes it hard to rapidly worsen a service.
|
||||||
|
|
||||||
|
</blockquote>
|
||||||
|
{{<quotecaption partOfType="OpinionNewsArticle">}}
|
||||||
|
{{<indieweb-person first-name="Nathan" last-name="Myhrvold" url="http://www.nathanmyhrvold.com/" itemprop="author">}},
|
||||||
|
{{<cited-work name="The Virtue of Inefficient Government" url="https://slate.com/news-and-politics/1996/10/the-virtue-of-inefficient-government.html" extraName="headline">}}
|
||||||
|
{{</quotecaption>}}
|
||||||
|
{{</quotation>}}
|
||||||
|
|
||||||
Ac­knowledge­ments {#acknowledgements}
|
Ac­knowledge­ments {#acknowledgements}
|
||||||
---------------------
|
--------------------------
|
||||||
|
|
||||||
{{<indieweb-person first-name="Denver" last-name="Gingerich" url="https://ossguy.com/">}} helped me brainstorm early in the writing process and provided useful information for the section about XMPP.
|
{{<indieweb-person itemprop="mentions" first-name="Denver" last-name="Gingerich" url="https://ossguy.com/">}} helped me brainstorm early in the writing process and provided useful information for the section about XMPP.
|
||||||
|
|
||||||
Thanks to {{<indieweb-person first-name="Barna" last-name="Zsombor" url="https://bzsombor.web.elte.hu/">}} and carbolymer for giving good feedback over IRC.
|
Thanks to {{<indieweb-person itemprop="mentions" first-name="Barna" last-name="Zsombor" url="https://bzsombor.web.elte.hu/">}} and carbolymer for giving good feedback over IRC.
|
||||||
|
|
||||||
|
|
||||||
[^1]: [This Hacker News comment](https://news.ycombinator.com/item?id=25961895) in particular planted quite a few seeds for this article.
|
[^1]: [This Hacker News comment](https://news.ycombinator.com/item?id=25961895) in particular planted quite a few seeds for this article.
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,6 @@
|
||||||
---
|
---
|
||||||
date: "2021-01-12T00:03:10-08:00"
|
date: "2021-01-12T00:03:10-08:00"
|
||||||
|
lastmod: "2021-12-01T22:01:21-08:00"
|
||||||
description: Using thermal physics, cosmology, and computer science to calculate password vulnerability to the biggest possible brute-force attack.
|
description: Using thermal physics, cosmology, and computer science to calculate password vulnerability to the biggest possible brute-force attack.
|
||||||
outputs:
|
outputs:
|
||||||
- html
|
- html
|
||||||
|
|
@ -10,21 +11,37 @@ tags:
|
||||||
title: Becoming physically immune to brute-force attacks
|
title: Becoming physically immune to brute-force attacks
|
||||||
footnote_heading: References and endnotes
|
footnote_heading: References and endnotes
|
||||||
---
|
---
|
||||||
|
<section role="doc-preface">
|
||||||
|
|
||||||
|
<h2 id="preface">Preface</h2>
|
||||||
|
|
||||||
This is a tale of the intersection between thermal physics, cosmology, and a tiny amount of computer science to answer a seemingly innocuous question: "How strong does a password need to be for it to be physically impossible to brute-force, ever?"
|
This is a tale of the intersection between thermal physics, cosmology, and a tiny amount of computer science to answer a seemingly innocuous question: "How strong does a password need to be for it to be physically impossible to brute-force, ever?"
|
||||||
|
|
||||||
[TLDR](#conclusion-tldr) at the bottom.
|
[TLDR](#conclusion-tldr) at the bottom.
|
||||||
|
|
||||||
|
<div role="doc-tip">
|
||||||
|
|
||||||
_Note: this post contains equations. Since none of the equations were long or complex, I decided to just write them out in code blocks instead of using images or MathML the way Wikipedia does._
|
_Note: this post contains equations. Since none of the equations were long or complex, I decided to just write them out in code blocks instead of using images or MathML the way Wikipedia does._
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
_Update: I implemented the ideas in this blog post (and more) in a program/library, [MOAC](https://sr.ht/~seirdy/MOAC/)_
|
_Update: I implemented the ideas in this blog post (and more) in a program/library, [MOAC](https://sr.ht/~seirdy/MOAC/)_
|
||||||
|
|
||||||
Introduction
|
</section>
|
||||||
------------
|
|
||||||
|
{{<toc>}}
|
||||||
|
|
||||||
|
<section role="doc-introduction">
|
||||||
|
|
||||||
|
Intro­duction {#introduction}
|
||||||
|
-----------------
|
||||||
|
|
||||||
I realize that advice on password strength can get outdated. As supercomputers grow more powerful, password strength recommendations need to be updated to resist stronger brute-force attacks. Passwords that are strong today might be weak in the future. **How long should a password be in order for it to be physically impossible to brute-force, ever?**
|
I realize that advice on password strength can get outdated. As supercomputers grow more powerful, password strength recommendations need to be updated to resist stronger brute-force attacks. Passwords that are strong today might be weak in the future. **How long should a password be in order for it to be physically impossible to brute-force, ever?**
|
||||||
|
|
||||||
This question might not be especially practical, but it's fun to analyze and offers interesting perspective regarding sane upper-limits on password strength.
|
This question might not be especially practical, but it's fun to analyze and offers interesting perspective regarding sane upper-limits on password strength.
|
||||||
|
|
||||||
|
</section>
|
||||||
|
|
||||||
Asking the right question
|
Asking the right question
|
||||||
-------------------------
|
-------------------------
|
||||||
|
|
||||||
|
|
@ -79,7 +96,7 @@ Obviously, I'm not taking into account future mathematical advances; my crystal
|
||||||
Finally, there's always a non-zero probability of a brute-force attack guessing a password with a given entropy. Literal "immunity" is impossible. Lowering this probability to statistical insignificance renders our password practically immune to brute-force attacks.
|
Finally, there's always a non-zero probability of a brute-force attack guessing a password with a given entropy. Literal "immunity" is impossible. Lowering this probability to statistical insignificance renders our password practically immune to brute-force attacks.
|
||||||
|
|
||||||
Compu­tation
|
Compu­tation
|
||||||
-----------
|
----------------
|
||||||
|
|
||||||
How much energy does MOAC use per guess during a brute-force attack? In the context of this thought experiment, this number should be unrealistically low. I settled on [<var>k</var><var>T</var>](https://en.wikipedia.org/wiki/KT_(energy)). <var>k</var> represents the [Boltzmann Constant](https://en.wikipedia.org/wiki/Boltzmann_constant) (about 1.381×10<sup>-23</sup> J/K) and <var>T</var> represents the temperature of the system. Their product corresponds to the amount of heat required to create a 1 nat increase in a system's entropy.
|
How much energy does MOAC use per guess during a brute-force attack? In the context of this thought experiment, this number should be unrealistically low. I settled on [<var>k</var><var>T</var>](https://en.wikipedia.org/wiki/KT_(energy)). <var>k</var> represents the [Boltzmann Constant](https://en.wikipedia.org/wiki/Boltzmann_constant) (about 1.381×10<sup>-23</sup> J/K) and <var>T</var> represents the temperature of the system. Their product corresponds to the amount of heat required to create a 1 nat increase in a system's entropy.
|
||||||
|
|
||||||
|
|
@ -213,7 +230,7 @@ An excerpt from a religious text with a trailing space:
|
||||||
Don't use actual excerpts from pre-existing works as your password.
|
Don't use actual excerpts from pre-existing works as your password.
|
||||||
|
|
||||||
Conclusion, TLDR
|
Conclusion, TLDR
|
||||||
---------------
|
----------------
|
||||||
|
|
||||||
Question: How much entropy should a password have to ensure it will _never_ be vulnerable to a brute-force attack? Can an impossibly efficient computer--the MOAC--crack your password?
|
Question: How much entropy should a password have to ensure it will _never_ be vulnerable to a brute-force attack? Can an impossibly efficient computer--the MOAC--crack your password?
|
||||||
|
|
||||||
|
|
@ -237,28 +254,24 @@ This article's approach deliberately disregards computation speed, focusing only
|
||||||
|
|
||||||
One well-known approach to calculating physical limits of computation is [Bremermann's limit](https://en.wikipedia.org/wiki/Bremermann%27s_limit), which calculates the speed of computation given a finite amount of mass. This article's approach disregards time, focusing only on mass-energy equivalence.
|
One well-known approach to calculating physical limits of computation is [Bremermann's limit](https://en.wikipedia.org/wiki/Bremermann%27s_limit), which calculates the speed of computation given a finite amount of mass. This article's approach disregards time, focusing only on mass-energy equivalence.
|
||||||
|
|
||||||
[A publication](https://arxiv.org/abs/quant-ph/9908043)[^5] by Seth Lloyd from MIT further explores limits to computation speed on an ideal 1-kilogram computer.
|
<span itemprop="mentions" itemscope itemtype="https://schema.org/ScholarlyArticle">{{<cited-work name="Ultimate physical limits to computation" extraName="headline" url="https://arxiv.org/abs/quant-ph/9908043">}} by {{<indieweb-person itemprop="author" first-name="Seth" last-name="Lloyd" url="https://meche.mit.edu/people/faculty/SLLOYD@MIT.EDU">}}</span> further explores limits to computation speed on an ideal 1-kilogram computer.
|
||||||
|
|
||||||
Ac­knowledge­ments {#acknowledgements}
|
Ac­knowledge­ments {#acknowledgements}
|
||||||
---------------------
|
--------------------------
|
||||||
|
|
||||||
Thanks to [Barna Zsombor](https://bzsombor.web.elte.hu/) and [Ryan Coyler](https://rcolyer.net/) for helping me over IRC with my shaky physics and pointing out the caveats of my approach. u/RisenSteam on Reddit also corrected an incorrect reference to AES-256 encryption by bringing up salts.
|
Thanks to {{<indieweb-person itemprop="mentions" first-name="Barna" last-name="Zsombor" url="https://bzsombor.web.elte.hu/">}} and {{<indieweb-person itemprop="mentions" first-name="Ryan" last-name="Coyler" url="https://rcolyer.net/">}} for helping me over IRC with my shaky physics and pointing out the caveats of my approach. u/RisenSteam on Reddit also corrected an incorrect reference to AES-256 encryption by bringing up salts.
|
||||||
|
|
||||||
My notes from Thermal Physics weren't enough to write this; various Wikipedia articles were also quite helpful, most of which were linked in the body of the article.
|
My notes from Thermal Physics weren't enough to write this; various Wikipedia articles were also quite helpful, most of which were linked in the body of the article.
|
||||||
|
|
||||||
While I was struggling to come up with a good expression for the minimum energy used per password guess, I stumbled upon a [blog post](https://www.schneier.com/blog/archives/2009/09/the_doghouse_cr.html) by Bruce Schneier. It contained a useful excerpt from his book _Applied Cryptography_[^6] involving setting the minimum energy per computation to <var>k</var><var>T</var>. I chose a more conservative estimate for <var>T</var> than Schneier did, and a _much_ greater source of energy.
|
While I was struggling to come up with a good expression for the minimum energy used per password guess, I stumbled upon <span itemprop="mentions" itemscope itemtype="https://schema.org/Book">{{<cited-work name="Applied Cryptography" url="https://www.schneier.com/books/applied-cryptography/">}} by {{<indieweb-person first-name="Bruce" last-name="Schneier" url="https://schneier.com/">}}</span>. An excerpt [on his blog](https://www.schneier.com/blog/archives/2009/09/the_doghouse_cr.html) involving setting the minimum energy per computation to <var>k</var><var>T</var>. I chose a more conservative estimate for <var>T</var> than Schneier did, and a _much_ greater source of energy.
|
||||||
|
|
||||||
|
|
||||||
[^1]: James Massey (1994). "Guessing and entropy" (PDF). Proceedings of 1994 IEEE International Symposium on Information Theory. IEEE. p. 204.
|
[^1]: <span itemprop="citation">James Massey (1994). "Guessing and entropy" (PDF). Proceedings of 1994 IEEE International Symposium on Information Theory. IEEE. p. 204.</span>
|
||||||
|
|
||||||
[^2]: Assis, A. K. T.; Neves, M. C. D. (3 July 1995). "History of the 2.7 K Temperature Prior to Penzias and Wilson"
|
[^2]: <span itemprop="citation">Assis, A. K. T.; Neves, M. C. D. (3 July 1995). "History of the 2.7 K Temperature Prior to Penzias and Wilson"</span>
|
||||||
|
|
||||||
[^3]: The MOAC 2 was supposed to be able to consume other sources of energy such as dark matter and dark energy. Unfortunately, Intergalactic Business Machines ran out of funds since all their previous funds, being made of matter, were consumed by the original MOAC.
|
[^3]: The MOAC 2 was supposed to be able to consume other sources of energy such as dark matter and dark energy. Unfortunately, Intergalactic Business Machines ran out of funds since all their previous funds, being made of matter, were consumed by the original MOAC.
|
||||||
|
|
||||||
[^4]: This is a massive oversimplification; there isn't a single answer to the question "What is the volume of the observable universe?" Using this speed-of-light approach is one of multiple valid perspectives. The absolute size of the observable universe is much greater due to the way expansion works, but stuffing that into the MOAC's furnace would require moving mass faster than the speed of light.
|
[^4]: This is a massive oversimplification; there isn't a single answer to the question "What is the volume of the observable universe?" Using this speed-of-light approach is one of multiple valid perspectives. The absolute size of the observable universe is much greater due to the way expansion works, but stuffing that into the MOAC's furnace would require moving mass faster than the speed of light.
|
||||||
|
|
||||||
[^5]: Lloyd, S., "Ultimate Physical Limits to Computation," Nature 406.6799, 1047-1054, 2000.
|
|
||||||
|
|
||||||
[^6]: Schneier, Bruce. Applied Cryptography, Second Edition, John Wiley & Sons, 1996.
|
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -1,5 +1,6 @@
|
||||||
---
|
---
|
||||||
date: "2021-04-16T17:28:06-07:00"
|
date: "2021-04-16T17:28:06-07:00"
|
||||||
|
lastmod: "2021-06-11T15:08:29-07:00"
|
||||||
description: Recently, people have been telling webmasters to add a Permissions-Policy
|
description: Recently, people have been telling webmasters to add a Permissions-Policy
|
||||||
header to their sites to opt out of FLoC. The reality of the situation isn't so
|
header to their sites to opt out of FLoC. The reality of the situation isn't so
|
||||||
simple.
|
simple.
|
||||||
|
|
|
||||||
|
|
@ -13,6 +13,11 @@ footnote_heading: Notes
|
||||||
featured: 1
|
featured: 1
|
||||||
title: A look at search engines with their own indexes
|
title: A look at search engines with their own indexes
|
||||||
---
|
---
|
||||||
|
|
||||||
|
<section role="doc-preface">
|
||||||
|
|
||||||
|
<h2 id="preface">Preface</h2>
|
||||||
|
|
||||||
This is a cursory review of all the indexing search engines I have been able to find.
|
This is a cursory review of all the indexing search engines I have been able to find.
|
||||||
|
|
||||||
The three dominant English search engines with their own indexes[^1] are Google, Bing, and Yandex (<abbr title="Google, Bing, Yandex">GBY</abbr>). Many alternatives to GBY exist, but almost none of them have their own results; instead, they just source their results from GBY.
|
The three dominant English search engines with their own indexes[^1] are Google, Bing, and Yandex (<abbr title="Google, Bing, Yandex">GBY</abbr>). Many alternatives to GBY exist, but almost none of them have their own results; instead, they just source their results from GBY.
|
||||||
|
|
@ -23,6 +28,8 @@ This page is a "living document" that I plan on updating indefinitely. Check for
|
||||||
|
|
||||||
I plan on updating the engines in the top two categories with more info comparing the structured/linked data the engines leverage (RDFa vocabularies, microdata, microformats, JSON-LD, etc.) to help authors determine which formats to use.
|
I plan on updating the engines in the top two categories with more info comparing the structured/linked data the engines leverage (RDFa vocabularies, microdata, microformats, JSON-LD, etc.) to help authors determine which formats to use.
|
||||||
|
|
||||||
|
</section>
|
||||||
|
|
||||||
{{<toc>}}
|
{{<toc>}}
|
||||||
|
|
||||||
About the list
|
About the list
|
||||||
|
|
@ -45,14 +52,17 @@ These are large engines that pass all my standard tests and more.
|
||||||
|
|
||||||
- Google: the biggest index. Allows submitting pages and sitemaps for crawling, and [even supports WebSub](https://developers.google.com/search/docs/advanced/sitemaps/build-sitemap#addsitemap) to automate the process. Powers a few other engines:
|
- Google: the biggest index. Allows submitting pages and sitemaps for crawling, and [even supports WebSub](https://developers.google.com/search/docs/advanced/sitemaps/build-sitemap#addsitemap) to automate the process. Powers a few other engines:
|
||||||
|
|
||||||
- [Startpage](https://www.startpage.com/)
|
- [Startpage](https://www.startpage.com/), possibly the most popular Google proxy.
|
||||||
- [GMX Search](https://search.gmx.com/web)
|
|
||||||
|
- [GMX Search](https://search.gmx.com/web), run by a popular German email provider.
|
||||||
|
|
||||||
- (discon­tinued) Runnaroo
|
- (discon­tinued) Runnaroo
|
||||||
|
|
||||||
- [SAPO](https://www.sapo.pt/) (Portu­guese interface, can work with English results)
|
- [SAPO](https://www.sapo.pt/) (Portu­guese interface, can work with English results)
|
||||||
|
|
||||||
- Bing: the runner-up. Allows submitting pages and sitemaps for crawling without login using [the IndexNow API](https://www.indexnow.org/). Its index powers many other engines:
|
- Bing: the runner-up. Allows submitting pages and sitemaps for crawling without login using [the IndexNow API](https://www.indexnow.org/). Its index powers many other engines:
|
||||||
|
|
||||||
- Yahoo (and its sibling engine, [One­Search](https://www.yahoo.com/now/verizon-launches-search-engine-onesearch-132901132.html))
|
- Yahoo (and its sibling engine, One­Search)
|
||||||
- DuckDuck­Go[^2]
|
- DuckDuck­Go[^2]
|
||||||
- AOL
|
- AOL
|
||||||
- Qwant (partial)[^3]
|
- Qwant (partial)[^3]
|
||||||
|
|
@ -208,31 +218,46 @@ I'm unable to evaluate these engines properly since I don't speak the necessary
|
||||||
### Big indexes
|
### Big indexes
|
||||||
|
|
||||||
- Baidu: Chinese. Very large index; it's a major engine alongside GBY. Offers webmaster tools for site submission.
|
- Baidu: Chinese. Very large index; it's a major engine alongside GBY. Offers webmaster tools for site submission.
|
||||||
|
|
||||||
- Qihoo 360: Chinese. I'm not sure how independent this one is.
|
- Qihoo 360: Chinese. I'm not sure how independent this one is.
|
||||||
|
|
||||||
- Toutiao: Chinese. Not sure how independent this one is either.
|
- Toutiao: Chinese. Not sure how independent this one is either.
|
||||||
|
|
||||||
- Sogou: Chinese
|
- Sogou: Chinese
|
||||||
|
|
||||||
- Yisou: Chinese
|
- Yisou: Chinese
|
||||||
|
|
||||||
- [Naver](https://search.naver.com): Korean. Allows submitting sitemaps and feeds. Discovered via some Searx metasearch instances.
|
- [Naver](https://search.naver.com): Korean. Allows submitting sitemaps and feeds. Discovered via some Searx metasearch instances.
|
||||||
|
|
||||||
- [Seznam](https://www.seznam.cz/): Czech, seems relatively privacy-friendly. Discovered in the seirdy.one access logs. It allows site submission with webmaster tools.
|
- [Seznam](https://www.seznam.cz/): Czech, seems relatively privacy-friendly. Discovered in the seirdy.one access logs. It allows site submission with webmaster tools.
|
||||||
|
|
||||||
- [Cốc Cốc](https://coccoc.com/search): Vietnamese
|
- [Cốc Cốc](https://coccoc.com/search): Vietnamese
|
||||||
|
|
||||||
- [go.mail.ru](https://go.mail.ru/): Russian
|
- [go.mail.ru](https://go.mail.ru/): Russian
|
||||||
|
|
||||||
### Smaller indexes
|
### Smaller indexes
|
||||||
|
|
||||||
- [Vuhuv](https://www.vuhuv.com.tr/): Turkish. [alt domain](https://tr.vuhuv.com/)
|
- [Vuhuv](https://www.vuhuv.com.tr/): Turkish. [alt domain](https://tr.vuhuv.com/)
|
||||||
|
|
||||||
- [Parsijoo](https://www.parsijoo.ir/): Persian
|
- [Parsijoo](https://www.parsijoo.ir/): Persian
|
||||||
|
|
||||||
- [search.ch](https://www.search.ch): Regional search engine for Switzerland; users can restrict searches to their local regions.
|
- [search.ch](https://www.search.ch): Regional search engine for Switzerland; users can restrict searches to their local regions.
|
||||||
|
|
||||||
- [fastbot](https://www.fastbot.de/): German
|
- [fastbot](https://www.fastbot.de/): German
|
||||||
|
|
||||||
- [Moose.at](https://www.moose.at): German (Austria-based)
|
- [Moose.at](https://www.moose.at): German (Austria-based)
|
||||||
|
|
||||||
- [SOLOFIELD](https://solofield.net): Japanese
|
- [SOLOFIELD](https://solofield.net): Japanese
|
||||||
|
|
||||||
- [kaz.kz](http://kaz.kz): Kazakh and Russian, with a focus on "Kazakhstan's segment of the Internet"
|
- [kaz.kz](http://kaz.kz): Kazakh and Russian, with a focus on "Kazakhstan's segment of the Internet"
|
||||||
|
|
||||||
Misc
|
Misc
|
||||||
----
|
----
|
||||||
|
|
||||||
* Ask.com: The site is back. They claim to outsource search results. The results seem similar to Google, Bing, and Yandex; however, I can’t pinpoint exactly where their results are coming from. Also, several sites from the "ask.com network" such as directhit.com, info.com, and kensaq.com have uniqe-looking results.
|
- Ask.com: The site is back. They claim to outsource search results. The results seem similar to Google, Bing, and Yandex; however, I can’t pinpoint exactly where their results are coming from. Also, several sites from the "ask.com network" such as directhit.com, info.com, and kensaq.com have uniqe-looking results.
|
||||||
|
|
||||||
- Not evaluated: Apple's search. It's only accessible through a search widget in iOS and macOS and shows very few results. This might change; see the next section.
|
- Not evaluated: Apple's search. It's only accessible through a search widget in iOS and macOS and shows very few results. This might change; see the next section.
|
||||||
|
|
||||||
- Partially evaluated: [Infinity Search](https://infinitysearch.co): young, small index. It recently split into a paid offering with the main index and [Infinity Decentralized](https://infinitydecentralized.com/), the latter of which allows users to select from community-hosted crawlers. I managed to try it out before it became a paid offering, and it seemed decent; however, I wasn't able to run the tests listed in the "Methodology" section. Allows submitting URLs and sitemaps into a text box, no other work required.
|
- Partially evaluated: [Infinity Search](https://infinitysearch.co): young, small index. It recently split into a paid offering with the main index and [Infinity Decentralized](https://infinitydecentralized.com/), the latter of which allows users to select from community-hosted crawlers. I managed to try it out before it became a paid offering, and it seemed decent; however, I wasn't able to run the tests listed in the "Methodology" section. Allows submitting URLs and sitemaps into a text box, no other work required.
|
||||||
|
|
||||||
Upcoming engines
|
Upcoming engines
|
||||||
|
|
@ -308,9 +333,13 @@ I compared results for esoteric queries side-by-side; if the first 20 results we
|
||||||
I tried to pick queries that should have a good number of results and show variance between search engines. An incomplete selection of queries I tested:
|
I tried to pick queries that should have a good number of results and show variance between search engines. An incomplete selection of queries I tested:
|
||||||
|
|
||||||
- "vim", "emacs", "neovim", and "nvimrc": Search engines with relevant results for "nvimrc" typically have a big index. Finding relevant results for the text editors "vim" and "emacs" instead of other topics that share the name is a challenging task.
|
- "vim", "emacs", "neovim", and "nvimrc": Search engines with relevant results for "nvimrc" typically have a big index. Finding relevant results for the text editors "vim" and "emacs" instead of other topics that share the name is a challenging task.
|
||||||
|
|
||||||
- "vim cleaner": should return results related to a [line of cleaning products](https://en.wikipedia.org/wiki/Vim_%28cleaning_product%29) rather than the Correct Text Editor.
|
- "vim cleaner": should return results related to a [line of cleaning products](https://en.wikipedia.org/wiki/Vim_%28cleaning_product%29) rather than the Correct Text Editor.
|
||||||
|
|
||||||
- "Seirdy": My site is relatively low-traffic, but my nickname is pretty unique and visible on several of the highest-traffic sites out there.
|
- "Seirdy": My site is relatively low-traffic, but my nickname is pretty unique and visible on several of the highest-traffic sites out there.
|
||||||
|
|
||||||
- "Project London": a small movie made with volunteers and <abbr title="Free, Libre, Open-Source Software">FLOSS</abbr> without much advertising. If links related to the movie show up, the engine's really good.
|
- "Project London": a small movie made with volunteers and <abbr title="Free, Libre, Open-Source Software">FLOSS</abbr> without much advertising. If links related to the movie show up, the engine's really good.
|
||||||
|
|
||||||
- "oppenheimer": a name that could refer to many things. Without context, it should refer to the physicist who worked on the atomic bomb in Los Alamos. Other historical queries: "magna carta" (intermediate), "the prince" (very hard).
|
- "oppenheimer": a name that could refer to many things. Without context, it should refer to the physicist who worked on the atomic bomb in Los Alamos. Other historical queries: "magna carta" (intermediate), "the prince" (very hard).
|
||||||
|
|
||||||
Some less-mainstream engines have noticed this article, which is great! I've had excellent discussions with people who work on several of these engines. Unfortunately, this article's visibility also incentivizes some engines to optimize specifically for any methodology I describe. I've addressed this by keeping a long list of test queries to myself. The simple queries above are a decent starting point for simple quick evaluations, but I also test for common search operators, keyword length, and types of domain-specific jargon. I also use queries designed to pull up specific pages with varying levels of popularity and recency to gauge the size, scope, and growth of an index.
|
Some less-mainstream engines have noticed this article, which is great! I've had excellent discussions with people who work on several of these engines. Unfortunately, this article's visibility also incentivizes some engines to optimize specifically for any methodology I describe. I've addressed this by keeping a long list of test queries to myself. The simple queries above are a decent starting point for simple quick evaluations, but I also test for common search operators, keyword length, and types of domain-specific jargon. I also use queries designed to pull up specific pages with varying levels of popularity and recency to gauge the size, scope, and growth of an index.
|
||||||
|
|
@ -343,9 +372,9 @@ Ac­know­ledge­ments {#acknowledgements}
|
||||||
|
|
||||||
Some of this content came from the [Search Engine Map](https://www.searchenginemap.com/) and [Search Engine Party](https://searchengine.party/). A few web directories also proved useful.
|
Some of this content came from the [Search Engine Map](https://www.searchenginemap.com/) and [Search Engine Party](https://searchengine.party/). A few web directories also proved useful.
|
||||||
|
|
||||||
{{<indieweb-person first-name="Matt" last-name="Wells" url="https://gigablast.com/bio.html" org="Gigablast" org-url="https://gigablast.com/">}} also gave me some helpful information about GBY which I included in the "Rationale" section. He's written more about big tech in the [Gigablast blog](https://gigablast.com/blog.html).
|
{{<indieweb-person itemprop="mentions" first-name="Matt" last-name="Wells" url="https://gigablast.com/bio.html" org="Gigablast" org-url="https://gigablast.com/">}} also gave me some helpful information about GBY which I included in the "Rationale" section. He's written more about big tech in the [Gigablast blog](https://gigablast.com/blog.html).
|
||||||
|
|
||||||
{{<indieweb-person first-name="Nicholas" last-name="Ferrell" url="https://emucafe.club/channel/naferrell" org="The New Leaf Journal" org-url="https://thenewleafjournal.com/">}} wrote a [great post](https://thenewleafjournal.com/a-2021-list-of-alternative-search-engines-and-search-resources/) on alternative search engines. He also gave me some [useful details](https://lists.sr.ht/~seirdy/seirdy.one-comments/%3C20210618031450.rb2twu4ypek6vvl3%40rkumarlappie.attlocal.net%3E) about Seznam, Naver, Baidu, and Goo.
|
<span itemprop="mentions" itemscope itemtype="https://schema.org/BlogPosting">{{<cited-work name="A 2021 List of Alternative Search Engines and Search Resources" url="https://thenewleafjournal.com/a-2021-list-of-alternative-search-engines-and-search-resources/">}} by {{<indieweb-person itemprop="author" first-name="Nicholas" last-name="Ferrell" url="https://emucafe.club/channel/naferrell" org="The New Leaf Journal" org-url="https://thenewleafjournal.com/">}}</span> is a great post on alternative search engines. He also gave me some [useful details](https://lists.sr.ht/~seirdy/seirdy.one-comments/%3C20210618031450.rb2twu4ypek6vvl3%40rkumarlappie.attlocal.net%3E) about Seznam, Naver, Baidu, and Goo.
|
||||||
|
|
||||||
|
|
||||||
[^1]: Yes, "indexes" is an acceptable plural form of the word "index". The word "indices" sounds weird to me outside a math class.
|
[^1]: Yes, "indexes" is an acceptable plural form of the word "index". The word "indices" sounds weird to me outside a math class.
|
||||||
|
|
|
||||||
|
|
@ -864,7 +864,9 @@ Keep the source order, DOM order, and visual order identical to ensure consisten
|
||||||
[Guideline 2.4 Navigable](https://www.w3.org/TR/WCAG22/#navigable) of the WCAG lists multiple criteria related to identifying and skipping sections of your pages, and for good reason:
|
[Guideline 2.4 Navigable](https://www.w3.org/TR/WCAG22/#navigable) of the WCAG lists multiple criteria related to identifying and skipping sections of your pages, and for good reason:
|
||||||
|
|
||||||
* Users of [switch access controls](https://en.wikipedia.org/wiki/Switch_access) find it slow and frustrating to navigate long lists of focusable items.
|
* Users of [switch access controls](https://en.wikipedia.org/wiki/Switch_access) find it slow and frustrating to navigate long lists of focusable items.
|
||||||
|
|
||||||
* Screen readers make it difficult to consume poorly-organized content non-linearly.
|
* Screen readers make it difficult to consume poorly-organized content non-linearly.
|
||||||
|
|
||||||
* Clients that don't support CSS can't prioritize content using a supplied stylesheet.
|
* Clients that don't support CSS can't prioritize content using a supplied stylesheet.
|
||||||
|
|
||||||
The list goes on: nearly every reader reliant upon assistive technologies (<abbr title="assistive technology">AT</abbr>) struggles to skim through poorly-organized pages.
|
The list goes on: nearly every reader reliant upon assistive technologies (<abbr title="assistive technology">AT</abbr>) struggles to skim through poorly-organized pages.
|
||||||
|
|
@ -1052,6 +1054,7 @@ The edges of a touch screen are often tap-targets (the top edge might toggle nav
|
||||||
On lists without visible bullets, I dropped the default indentation. I had to find other ways to ensure adequate tap-target size and provide sufficient non-interactive space for readers with hand-tremors to scroll. Some examples:
|
On lists without visible bullets, I dropped the default indentation. I had to find other ways to ensure adequate tap-target size and provide sufficient non-interactive space for readers with hand-tremors to scroll. Some examples:
|
||||||
|
|
||||||
- The [webmention list](#webmentions) after this article separates links with timestamps and some paragraph spacing.
|
- The [webmention list](#webmentions) after this article separates links with timestamps and some paragraph spacing.
|
||||||
|
|
||||||
- The [homepage posts list](https://seirdy.one/#posts) and the list of related articles at the beginning of [one of my posts](https://seirdy.one/2022/02/02/floss-security.html) separates links with non-interactive text descriptions
|
- The [homepage posts list](https://seirdy.one/#posts) and the list of related articles at the beginning of [one of my posts](https://seirdy.one/2022/02/02/floss-security.html) separates links with non-interactive text descriptions
|
||||||
|
|
||||||
### Line spacing
|
### Line spacing
|
||||||
|
|
@ -1322,21 +1325,37 @@ Your page should easily pass the harshest of tests without any extra effort if i
|
||||||
These tests begin reasonably, but gradually grow absurd. Once again, use your judgement.
|
These tests begin reasonably, but gradually grow absurd. Once again, use your judgement.
|
||||||
|
|
||||||
1. Test in all three major browser engines: Blink, Gecko, and WebKit.
|
1. Test in all three major browser engines: Blink, Gecko, and WebKit.
|
||||||
|
|
||||||
2. Evaluate the heaviness and complexity of your scripts (if any) by testing with your browser's <abbr title="just-in-time">JIT</abbr> compilation disabled.[^20]
|
2. Evaluate the heaviness and complexity of your scripts (if any) by testing with your browser's <abbr title="just-in-time">JIT</abbr> compilation disabled.[^20]
|
||||||
|
|
||||||
3. Test using the Tor Browser's safest security level enabled (disables JS and other features).
|
3. Test using the Tor Browser's safest security level enabled (disables JS and other features).
|
||||||
|
|
||||||
4. Load just the HTML. No CSS, no images, etc. Try loading without inline CSS as well for good measure.
|
4. Load just the HTML. No CSS, no images, etc. Try loading without inline CSS as well for good measure.
|
||||||
|
|
||||||
5. Print some pages in black-and-white, preferably with a simple laser printer.
|
5. Print some pages in black-and-white, preferably with a simple laser printer.
|
||||||
|
|
||||||
6. Test with assistive technologies such as screen readers, magnifiers, and switch controls.
|
6. Test with assistive technologies such as screen readers, magnifiers, and switch controls.
|
||||||
|
|
||||||
7. Ensure the page responds correctly to browser zoom. No sizes or dimensions should remain "fixed" across zoom levels.
|
7. Ensure the page responds correctly to browser zoom. No sizes or dimensions should remain "fixed" across zoom levels.
|
||||||
|
|
||||||
8. Test keyboard navigability with the <kbd>Tab</kbd> key and [caret navigation](https://en.wikipedia.org/wiki/Caret_navigation). Even without specifying tab indexes, tab selection should follow a logical order if you keep the layout simple.
|
8. Test keyboard navigability with the <kbd>Tab</kbd> key and [caret navigation](https://en.wikipedia.org/wiki/Caret_navigation). Even without specifying tab indexes, tab selection should follow a logical order if you keep the layout simple.
|
||||||
|
|
||||||
9. Test in textual browsers: lynx, links, w3m, ELinks, edbrowse, EWW, Netrik, etc.
|
9. Test in textual browsers: lynx, links, w3m, ELinks, edbrowse, EWW, Netrik, etc.
|
||||||
|
|
||||||
10. Test in an online website translator tool.
|
10. Test in an online website translator tool.
|
||||||
|
|
||||||
11. Read the (prettified and indented) HTML source itself and parse it with your brain. See if anything seems illogical or un­necessary. Imagine giving someone a printout of your page's `<body>` along with a whiteboard. If they have a basic knowledge of HTML tags, would they be able to draw something resembling your website?
|
11. Read the (prettified and indented) HTML source itself and parse it with your brain. See if anything seems illogical or un­necessary. Imagine giving someone a printout of your page's `<body>` along with a whiteboard. If they have a basic knowledge of HTML tags, would they be able to draw something resembling your website?
|
||||||
|
|
||||||
12. Test with unorthodox graphical browser engines, like NetSurf, Dillo, Servo, or the Serenity OS browser.
|
12. Test with unorthodox graphical browser engines, like NetSurf, Dillo, Servo, or the Serenity OS browser.
|
||||||
|
|
||||||
13. Test how your page renders in ancient browsers, like Netscape Navigator or Tkhtml. Use a TLS terminator or serve over HTTP from localhost.
|
13. Test how your page renders in ancient browsers, like Netscape Navigator or Tkhtml. Use a TLS terminator or serve over HTTP from localhost.
|
||||||
|
|
||||||
14. Try printing out your page in black-and-white from an unorthodox graphical browser.
|
14. Try printing out your page in black-and-white from an unorthodox graphical browser.
|
||||||
|
|
||||||
15. Download your webpage and test how multiple word processors render and generate PDFs from it.[^21]
|
15. Download your webpage and test how multiple word processors render and generate PDFs from it.[^21]
|
||||||
|
|
||||||
16. Combine conversion tools. Combine an HTML-<wbr>to-<wbr>EPUB converter and an EPUB-<wbr>to-<wbr>PDF converter, or stack multiple article-extraction utilities. Be creative and enjoy breaking your site. When something breaks, examine the breakage and see if it's caused by an issue in your markup, or a CSS feature with an equivalent alternative.
|
16. Combine conversion tools. Combine an HTML-<wbr>to-<wbr>EPUB converter and an EPUB-<wbr>to-<wbr>PDF converter, or stack multiple article-extraction utilities. Be creative and enjoy breaking your site. When something breaks, examine the breakage and see if it's caused by an issue in your markup, or a CSS feature with an equivalent alternative.
|
||||||
|
|
||||||
17. Build a time machine. Travel decades--or perhaps centuries--into the future. Keep going forward until the WWW is breathing its last breath. Test your site on future browsers. Figuring out how to transfer your files onto their computers might take some time, but you have a time machine so that shouldn't be too hard. When you finish, go back in time to [meet Benjamin Franklin](https://xkcd.com/567/).
|
17. Build a time machine. Travel decades--or perhaps centuries--into the future. Keep going forward until the WWW is breathing its last breath. Test your site on future browsers. Figuring out how to transfer your files onto their computers might take some time, but you have a time machine so that shouldn't be too hard. When you finish, go back in time to [meet Benjamin Franklin](https://xkcd.com/567/).
|
||||||
|
|
||||||
I'm still on step 16, trying to find new ways to break this page. If you come up with a new test, please [share it](mailto:~seirdy/seirdy.one-comments@lists.sr.ht).
|
I'm still on step 16, trying to find new ways to break this page. If you come up with a new test, please [share it](mailto:~seirdy/seirdy.one-comments@lists.sr.ht).
|
||||||
|
|
@ -1356,20 +1375,35 @@ Future updates
|
||||||
This article is, and will probably always be, an ongoing work-in-progress. Some areas I have yet to cover:
|
This article is, and will probably always be, an ongoing work-in-progress. Some areas I have yet to cover:
|
||||||
|
|
||||||
* How purely-cosmetic animations harm readers with learning and cognitive disabilities (e.g. attention disorders).
|
* How purely-cosmetic animations harm readers with learning and cognitive disabilities (e.g. attention disorders).
|
||||||
|
|
||||||
* How exposing new content on hover is inaccessible to users with magnifiers, hand tremors, switch access, and touch­screens.
|
* How exposing new content on hover is inaccessible to users with magnifiers, hand tremors, switch access, and touch­screens.
|
||||||
|
|
||||||
* Notes on improving support for braille displays.
|
* Notes on improving support for braille displays.
|
||||||
|
|
||||||
* How to work well with caret-based navigation.
|
* How to work well with caret-based navigation.
|
||||||
|
|
||||||
* How to choose phrasings such that some meaning can be inferred without understan­ding numbers, for [dyscalculic readers](https://en.wikipedia.org/wiki/Dyscalculia). This is more applicable to posts whose main focus is not mathematical or quantitative.
|
* How to choose phrasings such that some meaning can be inferred without understan­ding numbers, for [dyscalculic readers](https://en.wikipedia.org/wiki/Dyscalculia). This is more applicable to posts whose main focus is not mathematical or quantitative.
|
||||||
|
|
||||||
* How to include equations in a way that maximizes compatibility and accessibility.
|
* How to include equations in a way that maximizes compatibility and accessibility.
|
||||||
|
|
||||||
* Keypad-based navigation on feature phones (c.f. KaiOS devices).
|
* Keypad-based navigation on feature phones (c.f. KaiOS devices).
|
||||||
|
|
||||||
* How keyboard navigation can be altered by assistive tools such as screen readers.
|
* How keyboard navigation can be altered by assistive tools such as screen readers.
|
||||||
|
|
||||||
* How to avoid relying too much on formatting, for user agents that display unformatted text (e.g. textual feed readers like Newsboat)
|
* How to avoid relying too much on formatting, for user agents that display unformatted text (e.g. textual feed readers like Newsboat)
|
||||||
|
|
||||||
* Elaboration on how authors should delegate much of their formatting to the user agent, and how CSS resets are a symptom of a failure to do so.
|
* Elaboration on how authors should delegate much of their formatting to the user agent, and how CSS resets are a symptom of a failure to do so.
|
||||||
|
|
||||||
* Keyboard-driven browsers and extensions. Qutebrowser, Luakit, visurf, Tridactyl, etc.
|
* Keyboard-driven browsers and extensions. Qutebrowser, Luakit, visurf, Tridactyl, etc.
|
||||||
|
|
||||||
* Ways to support non-mainstream and older browsers by supporting subsets of specifications and using progressive enhance­ment.
|
* Ways to support non-mainstream and older browsers by supporting subsets of specifications and using progressive enhance­ment.
|
||||||
|
|
||||||
* Avoiding `_blank` targets in URLs unless absolutely necessary.
|
* Avoiding `_blank` targets in URLs unless absolutely necessary.
|
||||||
|
|
||||||
* Ways to improve compre­hension by readers who struggle to understand non-literal language (certain cognitive disabilities, non-native speakers unfamiliar with idioms, etc.). I might wait until the <abbr title="Web Accessibility Initiative">WAI</abbr> <cite>[Personali­zation Help and Support 1.0](https://w3c.github.io/personalization-semantics/help/index.html)</cite> draft specification matures and its vocabularies gain adoption before going in depth.
|
* Ways to improve compre­hension by readers who struggle to understand non-literal language (certain cognitive disabilities, non-native speakers unfamiliar with idioms, etc.). I might wait until the <abbr title="Web Accessibility Initiative">WAI</abbr> <cite>[Personali­zation Help and Support 1.0](https://w3c.github.io/personalization-semantics/help/index.html)</cite> draft specification matures and its vocabularies gain adoption before going in depth.
|
||||||
|
|
||||||
* Other accessible writing tips, maybe after I get a copy of <span itemprop="mentions" itemscope itemtype="https://schema.org/Book">{{<cited-work name="Writing Is Designing" url="https://rosenfeldmedia.com/books/writing-is-designing/">}} by {{<indieweb-person first-name="Michael" last-name="Metts" url="https://mjmetts.com/" itemprop="author">}} and {{<indieweb-person first-name="Andy" last-name="Welfe" url="https://www.andy.wtf/" itemprop="author">}}</span>. A relevant excerpt on writing accessibly is [on A List Apart](https://alistapart.com/article/standards-for-writing-accessibly/).
|
* Other accessible writing tips, maybe after I get a copy of <span itemprop="mentions" itemscope itemtype="https://schema.org/Book">{{<cited-work name="Writing Is Designing" url="https://rosenfeldmedia.com/books/writing-is-designing/">}} by {{<indieweb-person first-name="Michael" last-name="Metts" url="https://mjmetts.com/" itemprop="author">}} and {{<indieweb-person first-name="Andy" last-name="Welfe" url="https://www.andy.wtf/" itemprop="author">}}</span>. A relevant excerpt on writing accessibly is [on A List Apart](https://alistapart.com/article/standards-for-writing-accessibly/).
|
||||||
|
|
||||||
* Rules for descriptive link text, for screen reader navigation and for user-agents that display links as footnotes (e.g. some textual browsers with the `dump` flag).
|
* Rules for descriptive link text, for screen reader navigation and for user-agents that display links as footnotes (e.g. some textual browsers with the `dump` flag).
|
||||||
|
|
||||||
Con­clusion {#conclusion}
|
Con­clusion {#conclusion}
|
||||||
|
|
|
||||||
|
|
@ -1,30 +1,44 @@
|
||||||
---
|
---
|
||||||
date: "2021-01-27T16:13:36-08:00"
|
date: "2021-01-27T16:13:36-08:00"
|
||||||
description: WhatsApp's rise and recent PR efforts highlight a class of business models
|
lastmod: "2021-12-13T15:26:03-08:00"
|
||||||
that I call "user domestication".
|
description: WhatsApp's rise and recent PR efforts highlight a class of business models that I call "user domestication".
|
||||||
footnote_heading: References and endnotes
|
footnote_heading: References and endnotes
|
||||||
outputs:
|
outputs:
|
||||||
- html
|
- html
|
||||||
- gemtext
|
- gemtext
|
||||||
tags:
|
tags:
|
||||||
- free software
|
- free software
|
||||||
- user domestication
|
- user domestication
|
||||||
- privacy
|
- privacy
|
||||||
- platforms
|
- platforms
|
||||||
featured: 3
|
featured: 3
|
||||||
title: WhatsApp and the domestication of users
|
title: WhatsApp and the domestication of users
|
||||||
---
|
---
|
||||||
|
<section role="doc-introduction">
|
||||||
|
|
||||||
*Update: I wrote a follow up: [Keeping platforms open](/2021/02/23/keeping-platforms-open.html). Check it out if you found this article interesting.*
|
Intro­duction {#introduction}
|
||||||
|
-----------------
|
||||||
|
|
||||||
|
<div role="note">
|
||||||
|
|
||||||
|
_Update: I wrote a follow up: [Keeping platforms open](/2021/02/23/keeping-platforms-open.html). Check it out if you found this article interesting._
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div itemprop="backstory">
|
||||||
|
|
||||||
I have never used WhatsApp, and never will. Despite this, I still feel the need to write an article about WhatsApp since it's the perfect case study to help understand a class of businesses models I call "user domestication". The domestication of users is high on my list of problems plaguing the human race, and is worth a detailed explanation.
|
I have never used WhatsApp, and never will. Despite this, I still feel the need to write an article about WhatsApp since it's the perfect case study to help understand a class of businesses models I call "user domestication". The domestication of users is high on my list of problems plaguing the human race, and is worth a detailed explanation.
|
||||||
|
|
||||||
|
</div>
|
||||||
|
|
||||||
WhatsApp wasn't the first instant messenger of its kind, and probably won't be the last. I simply chose to focus on WhatsApp since its recent privacy issues have made it a hot topic.
|
WhatsApp wasn't the first instant messenger of its kind, and probably won't be the last. I simply chose to focus on WhatsApp since its recent privacy issues have made it a hot topic.
|
||||||
|
|
||||||
With the meta-explanation out of the way, let us begin.
|
With the meta-explanation out of the way, let us begin.
|
||||||
|
|
||||||
|
</section>
|
||||||
|
|
||||||
Whats­App's rise {#whatsapps-rise}
|
Whats­App's rise {#whatsapps-rise}
|
||||||
---------------
|
--------------------
|
||||||
|
|
||||||
For those unfamiliar, WhatsApp is a tool that makes it convenient and easy to help Facebook further its core mission: the optimization and auctioning of human behavior (colloquially known as "targeted advertising"). It originally persuaded people to consent to this by allowing them to send text to each other over the Internet, something that was [already possible](https://en.wikipedia.org/wiki/Comparison_of_instant_messaging_protocols), and combining an easy-to-learn UI with successful marketing. It then expanded to include features such as free voice and video calls. Free calls helped it grow to become the de-facto communication platform in many regions. I'm stunned at its ubiquity every time I visit my extended family in India; I'm frequently greeted by looks of confusion when I remind them that I don't use WhatsApp.
|
For those unfamiliar, WhatsApp is a tool that makes it convenient and easy to help Facebook further its core mission: the optimization and auctioning of human behavior (colloquially known as "targeted advertising"). It originally persuaded people to consent to this by allowing them to send text to each other over the Internet, something that was [already possible](https://en.wikipedia.org/wiki/Comparison_of_instant_messaging_protocols), and combining an easy-to-learn UI with successful marketing. It then expanded to include features such as free voice and video calls. Free calls helped it grow to become the de-facto communication platform in many regions. I'm stunned at its ubiquity every time I visit my extended family in India; I'm frequently greeted by looks of confusion when I remind them that I don't use WhatsApp.
|
||||||
|
|
||||||
|
|
@ -68,10 +82,42 @@ Those of us who were sounding the alarm a few years ago experienced a brief mome
|
||||||
|
|
||||||
### An attempt at damage control
|
### An attempt at damage control
|
||||||
|
|
||||||
The bait-and-switch operation incurred backlash significant enough for a noticeable minority of users to actually migrate; this number turned out to be slightly more than the rounding error WhatsApp was likely expecting. In response, WhatsApp delayed the change and published the following ad:
|
The bait-and-switch operation incurred backlash significant enough for a noticeable minority of users to actually migrate; this number turned out to be slightly more than the rounding error WhatsApp was likely expecting. In response, WhatsApp delayed the change and published an ad to improve its image.
|
||||||
|
|
||||||
|
{{<transcribed-image id="whatsapp-ad">}}
|
||||||
|
|
||||||
|
#### <span itemprop="name">Whatsapp Ad</span> {#whatsapp-ad-hd}
|
||||||
|
|
||||||
|
{{< transcribed-image-figure id="whatsapp-ad" has-transcript="true" >}}
|
||||||
|
|
||||||
{{< picture name="whatsapp_ad" alt="WhatsApp ad describing data not collected" >}}
|
{{< picture name="whatsapp_ad" alt="WhatsApp ad describing data not collected" >}}
|
||||||
|
|
||||||
|
<figcaption itemprop="caption">
|
||||||
|
|
||||||
|
WhatsApp ad, taken from <span itemprop="encodesCreativeWork" itemscope itemtype="https://schema.org/WebPage">{{<cited-work name="Answering your questions about WhatsApp’s Privacy Policy" url="https://faq.whatsapp.com/general/security-and-privacy/answering-your-questions-about-whatsapps-privacy-policy">}} (archived: <a href="https://web.archive.org/web/20210121163337/https://faq.whatsapp.com/general/security-and-privacy/answering-your-questions-about-whatsapps-privacy-policy" itemprop="archivedAt">Wayback Machine <time datetime="2021-01-21T16:33:37+00:00">2021-01-21</time> snapshot</a>)</span>
|
||||||
|
|
||||||
|
</figcaption>
|
||||||
|
{{< /transcribed-image-figure >}}
|
||||||
|
{{< transcribed-image-transcript >}}
|
||||||
|
|
||||||
|
WhatsApp Protects and Secures Your Personal Messages.
|
||||||
|
|
||||||
|
- WhatsApp cannot see your personal messages or hear your calls and neither can Facebook.
|
||||||
|
|
||||||
|
- WhatsApp does not keep logs of who everyone is messaging or calling.
|
||||||
|
|
||||||
|
- WhatsApp cannot see your shared location and neither can Facebook.
|
||||||
|
|
||||||
|
- WhatsApp does not share your contacts with Facebook.
|
||||||
|
|
||||||
|
- WhatsApp groups remain private.
|
||||||
|
|
||||||
|
- You can set your messages to disappear.
|
||||||
|
|
||||||
|
- You can download your data.
|
||||||
|
|
||||||
|
{{< /transcribed-image-transcript >}} {{< /transcribed-image >}}
|
||||||
|
|
||||||
The ad lists various data that WhatsApp doesn't collect or share. Allaying data collection concerns by listing data _not_ collected is misleading. WhatsApp doesn't collect hair samples or retinal scans either; not collecting that information doesn't mean it respects privacy because it doesn't change the information WhatsApp _does_ collect.
|
The ad lists various data that WhatsApp doesn't collect or share. Allaying data collection concerns by listing data _not_ collected is misleading. WhatsApp doesn't collect hair samples or retinal scans either; not collecting that information doesn't mean it respects privacy because it doesn't change the information WhatsApp _does_ collect.
|
||||||
|
|
||||||
The ad denies "keep[ing] logs of who everyone is messaging or calling". Collecting data is not the same as "keeping logs"; it's possible for metadata to be fed into an algorithm before being discarded. A model can thus learn that two users call each other frequently without keeping logs of the metadata for each call. The fact that they specifically chose to phrase this line around logging implies that WhatsApp either already collects this class of data or has deliberately left the door open to collecting it in the future.
|
The ad denies "keep[ing] logs of who everyone is messaging or calling". Collecting data is not the same as "keeping logs"; it's possible for metadata to be fed into an algorithm before being discarded. A model can thus learn that two users call each other frequently without keeping logs of the metadata for each call. The fact that they specifically chose to phrase this line around logging implies that WhatsApp either already collects this class of data or has deliberately left the door open to collecting it in the future.
|
||||||
|
|
@ -172,18 +218,18 @@ Translations
|
||||||
|
|
||||||
Translations are always welcome.
|
Translations are always welcome.
|
||||||
|
|
||||||
{{<indieweb-person first-name="Евгений" last-name="Кузнецов" url="https://evgenykuznetsov.org/">}} translated this article to Russian: <a lang="ru" hreflang="ru" rel="alternate" href="https://evgenykuznetsov.org/posts/2021/domestication/">WhatsApp и одомашнивание пользователей</a>.
|
- <span itemprop="workTranslation" itemscope itemtype="https://schema.org/BlogPosting">{{<indieweb-person itemprop="author" first-name="Евгений" last-name="Кузнецов" url="https://evgenykuznetsov.org/">}} translated this article to <span itemprop="inLanguage">Russian</span>: {{<cited-work lang="ru" rel="alternate" url="https://evgenykuznetsov.org/posts/2021/domestication/" name="WhatsApp и одомашнивание пользователей">}}</span>.
|
||||||
|
|
||||||
The Framalang translators at [Framasoft](https://framasoft.org/) translated this article to French: <a lang="fr" hreflang="fr" rel="alternate" href="https://framablog.org/2021/02/04/utilisateurs-libres-ou-domestiques-whatsapp-et-les-autres/">WhatsApp et la domestication des utilisateurs</a>.
|
- <span itemprop="workTranslation" itemscope itemtype="https://schema.org/BlogPosting">The Framalang translators at [Framasoft](https://framasoft.org/) translated this article to <span itemprop="inLanguage">French</span>: {{<cited-work lang="fr" rel="alternate" url="https://framablog.org/2021/02/04/utilisateurs-libres-ou-domestiques-whatsapp-et-les-autres/" name="WhatsApp et la domestication des utilisateurs">}}</span>.
|
||||||
|
|
||||||
{{<indieweb-person nickname="Licaon Kter" url="https://web.archive.org/web/20210924154338/https://convorb.im/">}} translated this article to Romanian: <a lang="ro" hreflang="ro" rel="alternate" href="https://web.archive.org/web/20210924154306/convorb.im/post/2021/02/14/whatsapp-si-domesticirea-utilizatorilor.html">WhatsApp și domesticirea utilizatorilor</a>.
|
- <span itemprop="workTranslation" itemscope itemtype="https://schema.org/BlogPosting">{{<indieweb-person itemprop="author" nickname="Licaon Kter" url="https://web.archive.org/web/20210924154338/https://convorb.im/">}} translated this article to <span itemprop="inLanguage">Romanian</span>: {{<cited-work lang="ro" rel="alternate" url="https://web.archive.org/web/20210924154306/convorb.im/post/2021/02/14/whatsapp-si-domesticirea-utilizatorilor.html" name="WhatsApp și domesticirea utilizatorilo">}}</span>.
|
||||||
|
|
||||||
{{<indieweb-person first-name="David" last-name="Jimenez" url="https://sgfault.com">}} translated this article to Spanish: <a lang="es" hreflang="es" rel="alternate" href="https://sgfault.com/post/2021/02/2021-02-21-whatsapp-y-la-domesticacion-de-usuarios/">WhatsApp y la domesticación de usuarios</a>.
|
- <span itemprop="workTranslation" itemscope itemtype="https://schema.org/BlogPosting">{{<indieweb-person itemprop="author" first-name="David" last-name="Jimenez" url="https://sgfault.com">}} translated this article to <span itemprop="inLanguage">Spanish</span>: {{<cited-work lang="es" rel="alternate" url="https://sgfault.com/post/2021/02/2021-02-21-whatsapp-y-la-domesticacion-de-usuarios/" name="WhatsApp y la domesticación de usuario">}}</span>.
|
||||||
|
|
||||||
{{<indieweb-person nickname="Skariko" url="https://www.lealternative.net/author/skariko/">}} of [Le Alternative](https://lealternative.net/) translated this article to Italian: <a lang="it-IT" hreflang="it-IT" rel="alternate" href="https://www.lealternative.net/2021/12/13/whatsapp-e-laddomesticamento-degli-utenti/">WhatsApp e l’addomesticamento degli utenti</a>.
|
- <span itemprop="workTranslation" itemscope itemtype="https://schema.org/BlogPosting">{{<indieweb-person itemprop="author" nickname="Skariko" url="https://www.lealternative.net/author/skariko/" org="Le Alternative" org-url="https://lealternative.net/">}} translated this article to <span itemprop="inLanguage">Italian</span>: {{<cited-work lang="it-IT" rel="alternate" url="https://www.lealternative.net/2021/12/13/whatsapp-e-laddomesticamento-degli-utenti/" name="WhatsApp e l’addomesticamento degli utent">}}</span>.
|
||||||
|
|
||||||
|
|
||||||
[^1]: Pierotti, R.; Fogg, B. (2017). The First Domestication: How Wolves and Humans Coevolved. Yale University Press.
|
[^1]: <span itemprop="citation">Pierotti, R.; Fogg, B. (2017). The First Domestication: How Wolves and Humans Coevolved. Yale University Press.</span>
|
||||||
|
|
||||||
[^2]: Many within the free software movement dislike the term "open source" for a [number of reasons](https://www.gnu.org/philosophy/open-source-misses-the-point.html); others use the terms "free" and "open source" [interchangeably](https://drewdevault.com/2019/09/17/The-wrong-words-but-the-right-ideas.html). Finally, many vendors use the word "free" to refer to price rather than freedom, prompting some free software supporters to adopt the term "libre" instead. All this can be quite confusing, which is why I prefer acronyms like FLOSS to describe these terms' intersection.
|
[^2]: Many within the free software movement dislike the term "open source" for a [number of reasons](https://www.gnu.org/philosophy/open-source-misses-the-point.html); others use the terms "free" and "open source" [interchangeably](https://drewdevault.com/2019/09/17/The-wrong-words-but-the-right-ideas.html). Finally, many vendors use the word "free" to refer to price rather than freedom, prompting some free software supporters to adopt the term "libre" instead. All this can be quite confusing, which is why I prefer acronyms like FLOSS to describe these terms' intersection.
|
||||||
|
|
||||||
|
|
@ -191,3 +237,4 @@ The Framalang translators at [Framasoft](https://framasoft.org/) translated this
|
||||||
|
|
||||||
[^4]: Moxie's blog post generated many responses. Two good follow-ups are on [Linux Weekly News](https://lwn.net/Articles/687294/) and a [blog post](https://matrix.org/blog/2020/01/02/on-privacy-versus-freedom/) by Matrix.org
|
[^4]: Moxie's blog post generated many responses. Two good follow-ups are on [Linux Weekly News](https://lwn.net/Articles/687294/) and a [blog post](https://matrix.org/blog/2020/01/02/on-privacy-versus-freedom/) by Matrix.org
|
||||||
|
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -14,7 +14,9 @@ Scope
|
||||||
This privacy policy applies to the following services:
|
This privacy policy applies to the following services:
|
||||||
|
|
||||||
1. The Web site <https://seirdy.one>
|
1. The Web site <https://seirdy.one>
|
||||||
|
|
||||||
2. The hidden Web service [http://wgq3\[...\]d<wbr>.onion](http://wgq3bd2kqoybhstp77i3wrzbfnsyd27wt34psaja4grqiezqircorkyd.onion/ "{title='http://wgq3bd2kqoybhstp77i3wrzbfnsyd27wt34psaja4grqiezqircorkyd.onion'}"), accessible over the Tor network
|
2. The hidden Web service [http://wgq3\[...\]d<wbr>.onion](http://wgq3bd2kqoybhstp77i3wrzbfnsyd27wt34psaja4grqiezqircorkyd.onion/ "{title='http://wgq3bd2kqoybhstp77i3wrzbfnsyd27wt34psaja4grqiezqircorkyd.onion'}"), accessible over the Tor network
|
||||||
|
|
||||||
3. The Gemini capsule <gemini://seirdy.one>
|
3. The Gemini capsule <gemini://seirdy.one>
|
||||||
|
|
||||||
This policy only applies if served by one of those three services.
|
This policy only applies if served by one of those three services.
|
||||||
|
|
|
||||||
|
|
@ -1 +1 @@
|
||||||
<cite itemprop="name{{if .Get "extraName"}} {{ .Get "extraName" }}{{end}}" class="p-name"><a class="u-url" itemprop="url" href="{{ .Get "url" }}">{{ .Get "name" }}</a></cite>
|
<cite itemprop="name{{if .Get "extraName"}} {{ .Get "extraName" }}{{end}}" class="p-name"><a class="u-url" itemprop="url" href="{{ .Get "url" }}" {{- with .Get "lang" }} lang="{{ . }}" hreflang="{{ . }}"{{ end -}} {{- with .Get "rel" }} rel="{{ . }}"{{ end }}>{{ .Get "name" }}</a></cite>
|
||||||
Loading…
Reference in a new issue