Establish Website Requirements/Criteria

[freddy-m]

Following on from #977, we should create a bare bones template criteria to adapt for other sections. This will make deciding what to list far easier for both the team and for users wanting to contribute.

Here are my suggestions, largely taken from degoogle.

Must be open source (unless discusssed on a case-by-case basis).

• Open development, where the community can take place in submitting pull requests, and see development of the project
• Active development (non-translation based updates within the past 4 months)
• F-Droid (if applicable) or Direct Download source (such as GitHub releases)

Website uses HTTPS

• E2EE when applicable
• Modern encryption protocols

Privacy-centric

• Anonymous payments (cash, crytpocurrency, gift cards)
• Analytics must be privacy respecting (preferably self hosted)
• No Google reCaptcha, instead hCaptcha or other alternative

Secure

• Software has a good vunerability disclosure protocal/bug bounty program
• Service is encrypted

Extra (not required but cool)

• .onion address

---

Obviously, these are just suggestions and they should be discussed and modified to fit our needs.

[ignoramous]

Hi all:

F-Droid (if applicable) or Direct Download source (such as GitHub releases)

For Android apps claiming to be FOSS, F-Droid availability should be a must rather than optional. Since F-Droid supports reproducible builds, users can be sure that the app in its entirety is open-source and that the Github / Gitlab repo aren't just showpieces.

Analytics must be privacy respecting (preferably self hosted)

The web analytics industry has by and large moved server-side (see Netlify and Cloudflare Pages; and serverless Google Analytics that run on the Edge). And so, highlighting "no client-side analytics" is superfluous at best and misleading at-worst since web-properties that want to track their users have plenty tools and the means to do so, all the while fooling the naive among us with "no client-side analytics" rhetoric.

[Mikaela]

For Android apps claiming to be FOSS, F-Droid availability should be a must rather than optional. Since F-Droid supports reproducible builds, users can be sure that the app in its entirety is open-source and that the Github / Gitlab repo aren't just showpieces.

How many applications do support and use reproducible builds on F-Droid? Do you propose requiring availability from F-Droid or reproducible builds from the app?

[ignoramous]

How many applications do support and use reproducible builds on F-Droid? Do you propose requiring availability from F-Droid or reproducible builds from the app?

F-Droid has implemented reproducible builds. It isn't up to the apps. To be included in F-Droid's official repository, F-Droid needs to be able to build apps reproducibly from source. If an app is partially closed source or uses any proprietary bits whatsoever, F-Droid wouldn't build it. Granted there are some gaps in their implementation, but the F-Droid volunteers do constantly seek ways to weed out those anomalies.

[Mikaela]

To be included in F-Droid's official repository, F-Droid needs to be able to build apps reproducibly from source.

Could you link me to further reading on this subject? Their documentation on reproducible builds says that they are only supported, their merge request template points to App Inclusion Policy which again has no word on reproducible builds, so I don't think they currently demand it from apps.

[ignoramous]

Oh well, reproducible builds is a stricter criteria (and I thought I saw Izzy, a F-Droid volunteer, mention r-builds are enabled on some forum but can't remember where). Anyways, reproducible builds or not, F-Droid does have a stricter criteria for FOSS (ref). And so, FOSS apps, if they are indeed FOSS, should build with the F-Droid app-builder just fine.

[merodiro]

The web analytics industry has by and large moved server-side (see Netlify and Cloudflare Pages; and serverless Google Analytics that run on the Edge). And so, highlighting "no client-side analytics" is superfluous at best and misleading at-worst since web-properties that want to track their users have plenty tools and the means to do so, all the while fooling the naive among us with "no client-side analytics" rhetoric.

I don't think server-side tracking is as bad as client-side tracking. Client-side trackers can track every interaction you make on the website/app unlike server-side trackers which tracks fewer data

Privacy respecting self-hosted trackers or 0 client trackers are a step forward and shouldn't be ignored just because they can track you on the server

[freddy-m]

@privacyguides/team would love some more input on this if possible. The sooner we can get this done the better.

[SkewedZeppelin]

Instead of specifying a forge, specify an OSI approved license.
Too often people think a project/product is open source, when really they are just using GitHub as an issue tracker.
ie. Repo on GitHub != open source

Also maybe note the support channels/community of the projects? Things like Discord should be a negative. Whereas XMPP or Matrix should be a positive. And having e-mail available is nice to have for people who don't want to create an account to report bugs or submit patches.

[ignoramous]

Too often people think a project/product is open source, when really they are just using GitHub as an issue tracker.

There are also cases when only a part of the product / service is actually open-source.

[dngray]

Instead of specifying a forge, specify an OSI approved license.

I think what @freddy-m is getting at with that is that development is actually done in public. For example what we want to avoid is archives of source without any version control, or massive squashed commits for example.

We'd like to see issues/PRs in public.

[freddy-m]

Yep, @dngray is correct. Updated it to:

Open development, where the community can take place in submitting pull requests, and see development of the project

[ignoramous]

@freddy-m / @dngray

As long as code isn't obfuscated, availability of source code under OSI license is a criteria good enough, in my eyes.

Open development is a constraint that's not only hard to prove but a hassle for folks willing to share the code but not have the energy / motivation to manage community participation (NetGuard comes to mind). There are perfectly valid reasons to not develop in the open, one among them being harassed by copy-cats and disrespectful free loaders. Ex A: M66B/NetGuard@22e04a7#commitcomment-59167726

[freddy-m]

Would be worth making a general requirement that all software needs a good vunerability disclosure protocal/bug bounty program. cc @privacyguides/team

[samuel-lucas6]

Regarding file encryption tools, I would argue for the following requirements:

• Free and open source as specified, except the 4 month requirement should be done on a case by case basis. For example, if no bugs have been filed, and there's no need for new functionality, then it shouldn't automatically be delisted. Monocypher is a real world example where not much needs to be done
• Use modern and secure cryptographic algorithms
• Rely on a well-regarded cryptographic library (e.g. libsodium or Monocypher)
• Have a fully documented design for the purpose of peer review
• Have readable and self-documenting code for the purpose of peer review
• Have some documented method of reporting security vulnerabilities (e.g. a SECURITY.md file on GitHub)
• Have decent usage instructions or a tutorial
• Ideally, be secure by default without any options/customisation related to the cryptography (e.g. no tweaking algorithms and parameters such as key size, password hashing cost, etc)
• Ideally, have informative commits and a changelog to aid peer review
• Ideally, have digital signatures to verify the downloadable binaries.

Because of the documentation and readable code requirements, it means a project can be more easily evaluated from a security perspective to identify common vulnerabilities and bad practices without having to do a full code review, which few people are qualified to do.

[ph00lt0]

We might want to reference the recommendations of NIST (https://doi.org/10.6028/NIST.SP.800-57pt1r5), which are very well explained here: https://www.keylength.com/en/4/

[dngray]

I'm thinking these sound like very acceptable requirements. I would expect PGP, Veracrypt, LUKS (and Tomb) would stand up to these requirements.

• Kryptor uses age/minisign (I'm thinking about Add Age File Encryption privacyguides.org#293 maybe we won't directly add age as it is really a library, although you can use it on commandline it doesn't support signing, so would need a guide to go with it.)
• Cryptomator uses it's own library cryptolib which uses the javax.crypto API It was audited by Cure53 and has decent docs. On iOS it uses cryptolib-swift which uses Apple's CommonCrypto API. I don't think the swift version of cryptolib-swift was audited though, the audit report does make reference to the ObjectiveC implementation, which makes sense the audit report is a bit dated now 2017.

[samuel-lucas6]

Glad people seem to agree. In terms of those programs:

• GPG: I would argue it should only be recommended for signatures now. The official documentation is pretty unreadable, it supports algorithms that shouldn't really be used, the design is overcomplicated, there are lots of features that the average user won't use, there have been several vulnerabilities over the years, and so on.
• VeraCrypt: properly documented and has been audited, which fixed multiple vulnerabilities. However, it does allow the user to change the encryption algorithm and hash function, with some options not being recommended (e.g. RIPEMD-160, Streebog, and Whirlpool should not be used over SHA256 or SHA512). It actually includes Russian algorithms with dodgy S-boxes, so I suggest explaining that AES and SHA512 should be used. Cascade encryption should be discouraged.
• LUKS: this uses some better algorithms than VeraCrypt, is well documented, and is accessible in Linux distribution installers.
• Tomb: not looked into this, but if it makes LUKS easier to use, then that's a good thing.
• Kryptor: my tool, just to get that out of the way. It actually isn't compatible with age or Minisign because it uses different algorithms and file formats in attempt to improve on both tools. For some quick justification of changes, Minisign previously allowed signature forgery, XChaCha20-BLAKE2b offers more security than ChaCha20-Poly1305, Argon2 should be preferred over scrypt, age doesn't encrypt private keys, and age lacks sender authentication, meaning an attacker could replace the ciphertext without the recipient knowing. A full list of rationale is available on the website. My tool certainly isn't perfect either, but I'm hoping to fix a few things in v4.
• age: it's a better alternative to GPG for encryption. I would recommend listing it due to its popularity, Filippo Valsorda's reputation, and multiple recipient support. Plugins are meant to be coming out for things like YubiKeys too. The signature issue is unfortunate.
• Minisign: this should be listed due to its popularity, for being well documented, for using solid algorithms and parameters, for having a good tutorial, and due to Frank Denis' reputation.
• Cryptomator: very easy to use, has a GUI, well documented, has a good development team, uses Encrypt-then-MAC rather than an AEAD, and is very popular and should have received peer review.

Ideally, be secure by default without any options/customisation related to the cryptography (e.g. no tweaking algorithms and parameters such as key size, password hashing cost, etc)

If this was rigorously enforced, then GPG and VeraCrypt definitely wouldn't be recommended.

[dngray]

I've started an initial PR in privacyguides/privacyguides.org#822

[efb4f5ff-1298-471a-8973-3d47447115dc]

If the criteria are established for every page would this mean u guys are going to remove the worth mentioning section if they do not meet the criteria to be listed?

[samuel-lucas6]

Yes, I think the plan is to remove the Worth Mentioning sections all together.

[ph00lt0]

The tools should either be listed or removed. The worth mentioning contained a lot of stuff which isn't really recommendable. It is a good thing that the website is simplified and just gives the best options for people to obtain some level of privacy.

[dngray]

The tools should either be listed or removed. The worth mentioning contained a lot of stuff which isn't really recommendable.

This. We're doing exactly this and for that reason. You might have noticed it with some of the pages already.

[ghost]

Maybe it might be better to have requirements per section. Because with the current suggestions some already recommended services fail e.g. last time I checked, Signal uses reCaptcha and DuckDuckGo isn't open-source. In the case of DuckDuckGo (and other search engines) it doesn't matter much because you are still fully trusting the server.

[merodiro]

I think the open source requirement is for client-side software only. because there is no way to know if a server is running the exact code in a repo or no unless you self-host it

[samuel-lucas6]

Unless I'm mistaken, that's what's already happening. The top comments are just referring to a general template.

[freddy-m]

@samuel-lucas6 is right, it would just be nice to have a 'general template' so that we can adapt it for each section. Some sections don't need a seperate criteria because they aren't a particularly complex software, and thus are covered under a general template. Hope this makes sense.

[qqii]

I believe sucg strict requirements will be a detriment to overall utility of the site. With these restrictions put in place there's no point having a section of threat modeling only to not include the threat model uses to assess the approved applications. It also denies all users with a lesser model the curated comparison and comments.

I originally made a reddit post exploring my thoughts, here .