gourami

[UNMAINTAINED] Activitypub server in Rust
Log | Files | Refs | README | LICENSE

commit 012a4b704a5b46a7c85f13247ec4e8019d8086c5
parent abed74816269aa2dc5155c8e3046c35d1e09f9ec
Author: alex wennerberg <alex@alexwennerberg.com>
Date:   Sat, 16 May 2020 13:49:14 -0500

Code cleanup

Diffstat:
MREADME.md | 7++-----
Mdocs/ADMIN_GUIDE.md | 39+++++++++++++++++++++------------------
Mdocs/USER_GUIDE.md | 8++++++--
Mmigrations/2020-04-13-014917_initialize/up.sql | 3+--
Msrc/ap.rs | 135++++++-------------------------------------------------------------------------
Msrc/db/schema.rs | 2+-
Msrc/db/server_mutuals.rs | 12++++++------
Msrc/db/user.rs | 3++-
Msrc/lib.rs | 21++++++++++++---------
Msrc/main.rs | 17++++++++++++++---
Mtemplates/server_info.html | 24+++++++++++++++++++++++-
11 files changed, 97 insertions(+), 174 deletions(-)

diff --git a/README.md b/README.md @@ -1,7 +1,7 @@ # 🐟gourami ![Build and Test](https://github.com/alexwennerberg/gourami/workflows/Build%20and%20Test/badge.svg) -An intentionally small, ultra-lightweight social media network (ActivityPub integration TBD) +An intentionally small, ultra-lightweight ActivityPub social network ![image](docs/demo.jpg) @@ -11,15 +11,12 @@ Gourami differs from existing social networks in a number of ways: * **Intentionally small** -- designed to support 50-100 active users. I'm sure it could support more, but things could quickly become a mess. Gourami was hugely and directly inspired by the fantastic essay on [runyourown.social](https://runyourown.social) * **Invite-only and closed** -- a community curated by the server admin, rather than open to all. * **Free and open source** -- I find the privatization of the internet extremely concerning, especially the way that the very space for building community and networking with our friends is controlled by for-profit corporations with potentially different values and goals than their users. +* **Decentralized** -- Gourami uses [ActivityPub](https://activitypub.rocks/) for federation, but with an implementation that differs from existing ActivityPub servers such as Mastodon. Instances federate at the server level, rather than the user level, which means all users on the server share the same "neighborhood". * **A social network with physical context** -- Gourami should be easy to deploy in a physical space (such as a coffee shop or a local wireless network) or among people in a specific physical community, such as a school. In *How to Do Nothing*, Jenny Odell discusses the lack of a context, specifically physical and temporal context, in social media, and, while praising Mastodon, also calls for social networks that are tied to physical space. While Gourami does not force you to tie a deployment to a place, it is designed in such a way that such a deployment would be relatively easy. * **Extremely lightweight & fast** -- very little Javascript, plain text, small page sizes. Should run on extremely lightweight/cheap hardware and low-bandwidth networks. * **[Brutalist](https://brutalist-web.design/)** -- Stark and minimal, the design and interface should emphasize, rather than hide, the underlying building blocks of the web that comprise it. This will give Gourami a feel similar to 90s or 2000s web forums. * **Simple and feature-averse** -- A simpler Gourami is much easier for me to develop, support and maintain. I want Gourami to be reliable software that people can build communities on top of, and severely limiting the feature set makes that much easier. Once I get Gourami to a certain core feature set, my work will be dedicated to maintenance and care, rather than feature additions. This will allow people to develop long-term, stable social networks, and also develop forks without worrying about losing upstream changes. -Some goals of this project that are work in progress: -* Additional accessibility features -* RSS - ## Dependencies: * sqlite3 diff --git a/docs/ADMIN_GUIDE.md b/docs/ADMIN_GUIDE.md @@ -1,30 +1,42 @@ -(WIP -- it doesnt actually work like this yet) - # Admin Guide If you want to administer a Gourami server, you'll need a few technical skills: -1. Basic Linux sysadmin skills -- ability to set up a webserver. +1. Basic Linux sysadmin skills -- ability to set up a web server. 2. Basic SQL knowledge -- ability to query and insert records. Right now, Gourami does not have an admin interface or admin tools, so certain actions (such as resetting a user's password or deleting a post or account) will require manual SQL intervention. +## Installation and setup + +// TODO + ## Inviting users Gourami is invite-only. Right now, you create an invite by adding a record to the invitation_keys table and sharing that key with the user you're inviting. +## Connecting with other servers. + +Gourami uses ActivityPub to connect with other ActivityPub Actors. If you're familiar with ActivityPub, you should know that Gourami works somewhat differently than a service like Mastodon. + +Gourami connects through the "neighborhood" timeline. This means that any post that a user on your server makes in the neighborhood timeline is sent to all servers you are connected with. You can connect with either a server or an individual ActivityPub actor, such as a Mastodon user, but be aware that that user will see all posts in your neighborhood timeline. + +You will only be considered "connected" to a remote server if you follow that server and that server follows you back. + +Gourami doesn't implement unfollows yet, so you'll have to directly modify the database and communicate with the user / server you're unfollowing. + ## Social guidelines Gourami is built for small deployments -- I have not tested it or designed it for larger implementations. This gives users a lot more flexibility, but requires more trust on your end. For example, a user may be easily able to spam the timeline, spam everyone's notifications, DOS the server, etc, so as an admin you should only allow people on your instance that you trust. You will also find that the quality of the shared timelines will begin to degrade after too many users. If you're still interested in attempting a larger Gourami deployment, I won't stop you, but beware that you're in uncharted territory. I'm not big into formal rules or codes of context, but if you feel like that's important for your server, you may want to put it in your server message. -## Customizing Gourami -You may want to customize parts of Gourami, such as the CSS format or server message. Right now, html templates are compiled into the binary. In retrospect, it might have been a better idea to use a templating engine that is rendered at runtime. If you want to customize the html, you'll have to edit the file and recompile. I may move towards a different templating library at some point. +## Configuration +// TODO -## Securing your server +## Customizing Gourami -I would recommend following basic Linux syadmin best practices: disable password login, consider a hardened Linux distro, set up a firewall, etc. I'm not a security expert here, I would recommend following guides produced by those who are. +You may want to customize parts of Gourami, such as the CSS format or server message. Right now, html templates are compiled into the binary. In retrospect, it might have been a better idea to use a templating engine that is rendered at runtime. If you want to customize the html, you'll have to edit the file and recompile. I may move towards a different templating library at some point. ## Gourami's ActivityPub implementation @@ -32,22 +44,13 @@ Gourami's ActivityPub implementation is somewhat opinionated and a little esoter The server has a server actor. This is an ActivityPub actor of type "Organization" and is the only ActivityPub actor on the server. All requests go through this actor. This forces you to think of your server as a cohesive whole -- users or other servers can only follow an entire server, not individual users. I encourage you to think about how this would change the way you structure your community. -Currently, deletes are not supported. Deletes can be misleading in federation, and I think the simplest solution is just not to implement them. +Currently, deletes are not supported. The only audience supported for ingoing and outgoing messages is [public]. This both simplifies the AP implementation and, in my view, more accurately specifies how ActivityPub works in practice -- once I send my message to a remote server, there isn't really any guarantee as to where it will go. Most of these decisions were informed by simplicity -## Federation -- the "neighborhood" - -Use the admin command follow to follow a server. -That server must accept your follow, then follow you back in order to be in the neighborhood. There is no one-way following in gourami, we (ab)use the AP standard to force mutual follows - -## Passwordless local deployment - -Don't do this on the public internet, it is a bad idea and will only lead to ruin! Seriously, don't do it. - -## Federation +## Federation with non-gourami AP services ActivityPub varies across servers. Some functionality may not work with other AP servers. Examples of things that may break include: diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md @@ -6,9 +6,13 @@ You will get a notification when someone tags your username in a post or if they ## Creating a note -Enter your text in the note box, then click create note. The first post that you tag (via 📝X or >>X, where X is the post number) will be considered a post(s) you're replying to. #TODO implement +Enter your text in the note box, then click create note. The first post that you tag (via 📝X or >>X, where X is the post number) will be considered a post(s) you're replying to. -For every local post or user that you tag in a note, that post / user's owner will receive a notification. +For every local post or user that you tag in a note (using @), that user will receive a notification. + +HTML tags will be stripped from your note input, except for p, br, and span. + +(write about how remote notes are handled) ## Audiences diff --git a/migrations/2020-04-13-014917_initialize/up.sql b/migrations/2020-04-13-014917_initialize/up.sql @@ -9,9 +9,8 @@ CREATE TABLE users ( password VARCHAR(255), admin BOOLEAN default false, show_email BOOLEAN default false, - remote_url VARCHAR(100) + remote_user BOOLEAN default false ); -create unique index unique_remote on users(remote_url); CREATE UNIQUE INDEX users_username_idx ON users (username); CREATE UNIQUE INDEX users_email_idx ON users (email); diff --git a/src/ap.rs b/src/ap.rs @@ -192,7 +192,8 @@ pub fn process_create_note( let remote_username = ap_note.get_remote_user_name().unwrap_or(ap_note.attributed_to); // TODO -- prevent usernames iwth colons // strip out username let new_user = NewRemoteUser { - username: remote_username.clone() + username: remote_username.clone(), + remote_user: true, }; let new_user_id: i32 = conn.transaction(|| { @@ -213,7 +214,6 @@ pub fn process_create_note( remote_id: ap_note.id, remote_url: ap_note.url, }; - println!("{:?}", new_remote_note); insert_into(n::notes) .values(&new_remote_note) .execute(conn)?; @@ -276,11 +276,11 @@ pub async fn process_follow(v: Value) -> Result<(), Error> { // generate accept } -pub fn get_destinations() -> Vec<String> { +pub fn get_connected_remotes() -> Vec<ServerMutual> { // maybe lazy static this use crate::db::schema::server_mutuals::dsl::*; let conn = &POOL.get().unwrap(); - server_mutuals.select(inbox_url) + server_mutuals .filter(accepted.eq(true)) .filter(followed_back.eq(true)).load(conn).unwrap() } @@ -314,16 +314,15 @@ pub async fn get_remote_actor(actor_id: &str) -> Result<Actor, Error> { Ok(res) } -pub async fn unfollow_remote_server(remote_url: &str) -> Result<(), Error> { - Ok(()) -} - -pub async fn follow_remote_server(remote_url: &str) -> Result<(), Error> { +// TODO cleanup interface +pub async fn whitelist_or_follow_remote_server(remote_url: &str, send: bool) -> Result<(), Error> { let remote_actor: Actor = get_remote_actor(remote_url).await?; let inbox_url = &remote_actor.inbox; let actor_id = &remote_actor.id; let msg = generate_server_follow(actor_id, inbox_url)?; - send_ap_message(&msg, inbox_url.to_owned()).await?; + if send { + send_ap_message(&msg, inbox_url.to_owned()).await?; + } Ok(()) } @@ -407,7 +406,6 @@ impl HttpSignature for reqwest::RequestBuilder { req.url().path().to_string() }; let unsigned = config.begin_sign(req.method().as_str(), &path_and_query, bt).unwrap(); - println!("{:?}", &unsigned); let sig_header = unsigned .sign(server_key_id,|signing_string| { let private_key = read_file(Path::new(&env::var("SIGNATURE_PRIVKEY").unwrap())); @@ -424,14 +422,12 @@ impl HttpSignature for reqwest::RequestBuilder { ) .unwrap(); // let digest = digest::digest(&digest::SHA256, &signing_string.as_bytes()); - println!("{:?}", &signing_string); let hexencode = base64::encode(&signature); Ok(hexencode) as Result<_, Error> })? .signature_header(); // this SHOULD be OK // host and date? - println!("{:?}", &sig_header); let result = self.header("Signature", sig_header); println!("{:?}", &result); Ok(result) @@ -494,7 +490,6 @@ pub async fn verify_ap_message(method: &str, path_and_query: &str, headers: BTre let config = Config::default() .set_expiration(Duration::seconds(3600)) .dont_use_created_field(); - println!("{:?}", headers); let unverified = config .begin_verify(method, path_and_query, headers)?; let actor: Actor = get_remote_actor(unverified.key_id()).await?; @@ -507,117 +502,5 @@ pub async fn verify_ap_message(method: &str, path_and_query: &str, headers: BTre key.verify(signing_string.as_bytes(), &hexdecode).unwrap(); true }); - println!("{:?}", unverified); Ok(()) } - -#[cfg(test)] -mod tests { - use super::*; - - fn prepare_headers() -> HashMap<String, String> { - let mut headers = HashMap::new(); - headers.insert( - "Content-Type".to_owned(), - "application/activity+json".to_owned(), - ); - headers - } - - #[test] - fn test_verify_rsa() { - sign_and_verify_rsa( - Path::new(&env::var("SIGNATURE_PRIVKEY").unwrap()), - Path::new(&env::var("SIGNATURE_PUBKEY").unwrap()), - ) - .unwrap() - } - - #[test] - fn test_verify_ap_message() { - let mut headers = HashMap::new(); - headers.insert( - "Content-Type".to_owned(), - "application/activity+json".to_owned(), - ); - headers.insert( - "date".to_owned(), - "Fri, 08 May 2020 00:42:41 +0000".to_owned(), - ); - let sample = "keyId=\"http://localhost:3030/actor#key\",algorithm=\"hs2019\",headers=\"(request-target) content-type date\",signature=\"YCJ7bwIX8y6rJ9Be31wm4ZkiBqper4vGydPHc/avBRE7D7SpIfWO+aA00VQcHlAGYjNRLEWiA5SkpszW3wnAs5JzuRWK01pELsEluYyE54/ou+rc06DxPt9beb9mIrbPs9EByN6epkYAGuKna8xoE7qsjhpfz5Q0SfNP3qS10uLaP5/puFCxMVgDIb3wMiJz1WiCzWZ26e5Wujoea8l5HS37V4xYhqicXmTvU1SzEiC+Qsn3RteWTesItAEDID5CFOhFizkSvgYVNjpTMwbLf1QiqyfgctVQIYt4fuQSTlcdKjhpS1cAxKTJg5hFQ9vjo1Qm1NP6XBALcRWpAIw5SA==\""; - headers.insert("signature".to_owned(), sample.to_owned()); - verify_ap_message("post", "/inbox", headers); - } - - #[test] - fn test_send_ap() { - let body: Value = serde_json::from_str(r#"{"foo": "bar"}"#).unwrap(); - let req = reqwest::Client::new() - .post("http://localhost:3030/inbox") - // for mastodon config -- newer versions of httsig dont use this - .header("date", Utc::now().to_rfc2822()) - .json(&body) - .header( - "Content-Type", - "application/activity+json", - ) - .http_sign_outgoing() - .unwrap(); - } - - #[test] - fn test_empty_string() { - // to write - } - - #[test] // TODO -- set env variales in test - fn test_mastodon_create_status_example() { - let create_note_mastodon: Value = serde_json::from_str(r#"{ - "id": "https://mastodon.social/users/alexwennerberg/statuses/104028309437021899/activity", - "type": "Create", - "actor": "https://mastodon.social/users/alexwennerberg", - "published": "2020-04-20T01:27:10Z", - "to": [ - "https://www.w3.org/ns/activitystreams#Public" - ], - "cc": [ - "https://mastodon.social/users/alexwennerberg/followers" - ], - "object": { - "id": "https://mastodon.social/users/alexwennerberg/statuses/104028309437021899", - "type": "Note", - "summary": null, - "inReplyTo": null, - "published": "2020-04-20T01:27:10Z", - "url": "https://mastodon.social/@alexwennerberg/104028309437021899", - "attributedTo": "https://mastodon.social/users/alexwennerberg", - "to": [ - "https://www.w3.org/ns/activitystreams#Public" - ], - "cc": [ - "https://mastodon.social/users/alexwennerberg/followers" - ], - "sensitive": false, - "atomUri": "https://mastodon.social/users/alexwennerberg/statuses/104028309437021899", - "inReplyToAtomUri": null, - "conversation": "tag:mastodon.social,2020-04-20:objectId=167583625:objectType=Conversation", - "content": "hello world", - "contentMap": { - "en": "<p>&lt;a href=&quot;<a href=\"https://google.com\" rel=\"nofollow noopener noreferrer\" target=\"_blank\"><span class=\"invisible\">https://</span><span class=\"\">google.com</span><span class=\"invisible\"></span></a>&quot;&gt;hi&lt;/a&gt;</p>" - }, - "attachment": [], - "tag": [], - "replies": { - "id": "https://mastodon.social/users/alexwennerberg/statuses/104028309437021899/replies", - "type": "Collection", - "first": { - "type": "CollectionPage", - "next": "https://mastodon.social/users/alexwennerberg/statuses/104028309437021899/replies?only_other_accounts=true&page=true", - "partOf": "https://mastodon.social/users/alexwennerberg/statuses/104028309437021899/replies", - "items": [] - } - } - } - }"#).unwrap(); - } -} diff --git a/src/db/schema.rs b/src/db/schema.rs @@ -28,7 +28,7 @@ table! { password -> Nullable<Varchar>, admin -> Bool, show_email -> Bool, - remote_url -> Nullable<Varchar>, + remote_user -> Bool, } } diff --git a/src/db/server_mutuals.rs b/src/db/server_mutuals.rs @@ -3,12 +3,12 @@ use super::schema::server_mutuals; #[derive(Queryable, PartialEq, Debug)] pub struct ServerMutual { - id: i32, - actor_id: String, - accepted: bool, - followed_back: bool, - inbox_url: String, - outbox_url: String, // not implemented yet + pub id: i32, + pub actor_id: String, + pub inbox_url: String, + pub accepted: bool, + pub followed_back: bool, + pub outbox_url: Option<String>, // not implemented yet } #[derive(Insertable)] diff --git a/src/db/user.rs b/src/db/user.rs @@ -50,7 +50,7 @@ pub struct User { pub password: Option<String>, pub admin: bool, pub show_email: bool, - pub remote_url: Option<String>, + pub remote_user: bool, } impl User { @@ -102,6 +102,7 @@ pub struct NewUser<'a> { #[table_name = "users"] pub struct NewRemoteUser { pub username: String, + pub remote_user: bool, } // impl NewUser { diff --git a/src/lib.rs b/src/lib.rs @@ -24,6 +24,7 @@ use askama::Template; use db::conn::POOL; use db::note; use db::note::{Note, NoteInput}; +use db::server_mutuals::{ServerMutual}; use db::notification::{NewNotification, NewNotificationViewer, Notification, NotificationViewer}; use db::user::{NewUser, RegistrationKey, User, Username}; use diesel::insert_into; @@ -110,6 +111,7 @@ struct RegisterTemplate<'a> { struct ServerInfoTemplate<'a> { global: Global<'a>, users: Vec<User>, + server_mutuals: Vec<ServerMutual> } const PAGE_SIZE: i64 = 50; @@ -210,7 +212,9 @@ async fn handle_new_note_form(u: Option<User>, f: NewNoteRequest, sender: Unboun let n = new_note(&u, &f.note_input, f.neighborhood.is_some()).unwrap(); if n.neighborhood { let nj = ap::new_note_to_ap_message(&n, &u); - let destinations = ap::get_destinations(); + let destinations = ap::get_connected_remotes() + .into_iter() + .map(|s| s.inbox_url).collect(); sender.send((nj, destinations)).ok(); } let red_url: http::Uri = f.redirect_url.parse().unwrap(); @@ -359,12 +363,11 @@ fn do_register(form: RegisterForm, query_params: serde_json::Value) -> impl Repl password: &hash, email: &form.email, }; - // todo data validation insert_into(users).values(new_user).execute(conn).unwrap(); - // insert into database } } + // database // not good do_login(LoginForm { username: form.username, @@ -450,10 +453,8 @@ pub struct UserNote { } fn get_single_note(note_id: i32) -> Option<Vec<UserNote>> { - // doing some fancy recursive stuff + // Get note and all children, recursively let conn = &POOL.get().unwrap(); - // TODO -- it isnt serializing ids right - // Username is a hack because there are two ID columns and it doesnt work right let results: Vec<(Note, Username)> = diesel::sql_query(format!( r"with recursive tc( p ) as ( values({}) @@ -481,10 +482,10 @@ fn get_single_note(note_id: i32) -> Option<Vec<UserNote>> { ) } -fn get_users() -> Result<Vec<User>, diesel::result::Error> { +fn get_local_users() -> Result<Vec<User>, diesel::result::Error> { use db::schema::users::dsl as u; let conn = &POOL.get().unwrap(); - let users = u::users.load(conn); + let users = u::users.filter(u::remote_user.eq(false)).load(conn); users } @@ -620,10 +621,12 @@ impl<'a> Default for ErrorTemplate<'a> { } fn server_info_page(auth_user: Option<User>) -> impl Reply { - let users = get_users().unwrap(); + let users = get_local_users().unwrap(); + let server_mutuals = ap::get_connected_remotes(); render_template(&ServerInfoTemplate { global: Global::create(auth_user, "/server"), users: users, + server_mutuals: server_mutuals }) } diff --git a/src/main.rs b/src/main.rs @@ -14,17 +14,28 @@ async fn main() { .subcommand(App::new("run").about("Run server")) .subcommand(App::new("follow") .arg(Arg::with_name("URL") - .help("url of the remote server to follow") + .help("url of the AP actor to follow") .required(true) .index(1) ) ) + .subcommand(App::new("whitelist") + .arg(Arg::with_name("URL") + .help("url of the remote AP actor to whitelist. gourami will reject follows except from whitelisted AP actors. Following a remote actor also automatically whitelists that server.") + .required(true) + .index(1) + ) + ) + .get_matches(); - if let Some(m) = matches.subcommand_matches("run") { + if let Some(_) = matches.subcommand_matches("run") { run_server().await; } else if let Some(m) = matches.subcommand_matches("follow") { let url = m.value_of("URL").unwrap(); - ap::follow_remote_server(url).await.unwrap(); + ap::whitelist_or_follow_remote_server(url, true).await.unwrap(); + } else if let Some(m) = matches.subcommand_matches("whitelist") { + let url = m.value_of("URL").unwrap(); + ap::whitelist_or_follow_remote_server(url, false).await.unwrap(); } // reset password // follow remote diff --git a/templates/server_info.html b/templates/server_info.html @@ -1,17 +1,39 @@ {% extends "base.html" %} {% block content %} <div class="text-block padded"> - Welcome to Gourami, an intentionally small, private, minimalist, invite-only social network. See the <a href="https://github.com/alexwennerberg/gourami/blob/master/docs/USER_GUIDE.md">user guide</a> for more info or look at the <a href="https://github.com/alexwennerberg/gourami">project page</a>! Here is a list of users on this server -- maybe introduce yourself to someone you don't know! By default your email is not shared, but you can share it to other users on this instance on your edit user page. + Welcome to Gourami, an intentionally small, minimalist, invite-only <a href="https://activitypub.rocks/">ActivityPub</a>-based social network. See the <a href="https://github.com/alexwennerberg/gourami/blob/master/docs/USER_GUIDE.md">user guide</a> for more info or look at the <a href="https://github.com/alexwennerberg/gourami">project page</a>! This server can connect with other Gourami servers or other services that implement ActivityPub through the neighborhood. Right now, we are linked with the following actors: + <br> +<table> +{% for server in server_mutuals %} +<tr> + <td> + <b><a href="{{server.actor_id}}">{{server.actor_id}}</a></b> + </td> +</tr> +{%endfor%} +</table> + +<br> +Here is a list of users on this server -- maybe introduce yourself to someone you don't know! By default your email is not shared, but you can share it to other users on this instance on your edit user page. <br> <table> <tr> <th>username</th> + <th>email</th> <th>bio</th> </tr> {% for user in users %} <tr> <td><a href="/user/{{user.username}}">{{user.username}}</a> {%if user.admin %}<em>admin</em>{%endif%}</td> + <td> + {% match user.email %} + {% when Some with (e) %} + {% if user.show_email %} + <a href="mailto:{{e}}">{{e}}</a></td> + {% endif %} + {% when None %} + {%endmatch%} <td>{{user.bio}}</td> </tr> {% endfor %}