Sammlung von Newsfeeds

Xen & Databases

Ingo Jürgensmann - 13. Oktober 2018 - 21:21

I'm running PostgreSQL and MySQL on my server that both serve different databases to Wordpress, Drupal, Piwigo, Friendica, Mastodon, whatever...

In the past the databases where colocated in my mailserver VM whereas the webserver was running on a different VM. Somewhen I moved the databases from domU to dom0, maybe because I thought that the databases would be faster running on direct disk I/O in the dom0 environment, but can't remember the exact rasons anymore.

However, in the meantime the size of the databases grew and the number of the VMs did, too. MySQL and PostgreSQL are both configured/optimized to run with 16 GB of memory in dom0, but in the last months I experienced high disk I/O especially for MySQL and slow I/O performance in all the domU VMs because of that.

Currently iotop shows something like this:

Total DISK READ :     131.92 K/s | Total DISK WRITE :    1546.42 K/s
Actual DISK READ:     131.92 K/s | Actual DISK WRITE:       2.40 M/s
  TID  PRIO  USER     DISK READ  DISK WRITE  SWAPIN     IO>    COMMAND
 6424 be/4 mysql       0.00 B/s    0.00 B/s  0.00 % 60.90 % mysqld
18536 be/4 mysql      43.97 K/s   80.62 K/s  0.00 % 35.59 % mysqld
 6499 be/4 mysql       0.00 B/s   29.32 K/s  0.00 % 13.18 % mysqld
20117 be/4 mysql       0.00 B/s    3.66 K/s  0.00 % 12.30 % mysqld
 6482 be/4 mysql       0.00 B/s    0.00 B/s  0.00 % 10.04 % mysqld
 6495 be/4 mysql       0.00 B/s    3.66 K/s  0.00 % 10.02 % mysqld
20144 be/4 postgres    0.00 B/s   73.29 K/s  0.00 %  4.87 % postgres: hubzilla hubzi~
 2920 be/4 postgres    0.00 B/s 1209.28 K/s  0.00 %  3.52 % postgres: wal writer process
11759 be/4 mysql       0.00 B/s   25.65 K/s  0.00 %  0.83 % mysqld
18736 be/4 mysql       0.00 B/s   14.66 K/s  0.00 %  0.17 % mysqld
21768 be/4 root        0.00 B/s    0.00 B/s  0.00 %  0.02 % [kworker/1:0]
 2922 be/4 postgres    0.00 B/s   69.63 K/s  0.00 %  0.00 % postgres: stats collector process

MySQL data site is below configured max memory size for MySQL, so everything should more or less fit into memory. Yet, there is still a large amount of disk I/O by MySQL, much more than by PostgreSQL. Of course there is much I/O done by writes to the database.

However, I'm thinking of changing my setup again back to domU based database setup again, maybe one dedicated VM for both DBMS' or even two dedicated VMs for each of them? I'm not quite sure how Xen reacts to the current work load?

Back in the days when I did 3D computer graphic I did a lot of testing with different settings in regards of priorities and such. Basically one would think that giving the renderer more CPU time would speed of the rendering, but this turned out to be wrong: the higher the render tasks priority was, the slower the rendering got, because disk I/O (and other tasks that were necessary for the render task to work) got slowed down. When running the render task at lowest priority all the other necessary tasks could run on higher speed and return the CPU more quickly, which resulted in shorter render times.

So, maybe I experience something similar with the databases on dom0 here as well: dom0 is busy doing database work and this slows down all the other tasks (== domU VMs). When I would move databases back to domU this would enable dom0 again to better do its basic job of taking care of the domUs?

Of course, this is also a quite philosophical question, but what is the recommended setup? Is it better to separate the databases in two different VMs or just one? Or is running the databases on dom0 the best option?

I'm interested in your feedback, so please comment! :-)

Kategorie: DebianTags: DebianMySQLPostgreSQLXen 
Kategorien: Private Blogs

#Friendica vs #Hubzilla vs #Mastodon

Ingo Jürgensmann - 25. August 2018 - 18:58

I've been running a #Friendica node for several years now. Some months ago I also started to run a #Hubzilla hub as well. Some days ago I also installed #Mastodon on a virtual machine, because there was so much hype about Mastodon in the last days due to some changes Twitter made in regards of 3rd party clients.

All of those social networks do have their own focus:

Friendica: basically can connect to all other social networks, which is quite nice because there exists historically two different worlds: the Federation (Diaspora, Socialhome) and the Fediverse (GnuSocial, Mastodon, postActiv, Pleroma). Only Friendica and Hubzilla can federate with both: Federation and Fediverse.
Friendicas look&feel appears sometimes a little bit outdated and old, but it works very well and reliable.

Hubzilla: is the second player in the field of connecting both federations, but has a different focus. It is more of one-size-fits-all approach. If you need a microblogging site, a wiki, a cloud service, a website, etc. then Hubzilla is the way to go. The look&feel is a little bit more modern, but there are some quirks that appears a little odd to me. A unique feature for Hubzilla seems to be the concept of "nomadic accounts": you can move to a different hub and take all your data with you. Read more about that in the Hubzilla documentation.

Mastodon: this aims to be a replacement for Twitter as a microblogging service. It looks nice and shiny, has a bunch of nice clients for smartphones and has the largest userbase by far (which is not that important because of federation).
But the web GUI is rather limited and weird, as far as I can tell after just some days.

Technically spoken these are the main differences:
- Friendica: MySQL/MariaDB, PHP on the server, Clients: some Android clients, no iOS client
- Hubzilla: MySQL/MariaDB or PostgreSQL, PHP on the server, Clients: don't know, didn't care so far.
- Mastodon: PostgreSQL, Ruby on the server, Clients: many iOS and Android clients available

I'm not that big Ruby fan and if I remember correctly the Ruby stuff turned me away from Diaspora years ago and made me switch to Friendica, because back then it was a pain to maintain Diaspora. Mastodon addresses this by offering Docker container for the ease of installation and maintenance. But as I'm no Docker fan either, I followed the guide to install Mastodon without Docker, which works so far as well (for the last 3 days ).

So after all my Friendica node is still my favorit, because is just works and is reliable. Hubzilla has a different approach and offers a full set of webfeatures and nomadic accounts. The best I can say about Mastodon at this moment is: it runs on PostgreSQL and has nice clients on mobile devices.

Here are my instances:
- Friendica: https://nerdica.net/
- Hubzilla: https://silverhaze.eu/
- Mastodon: https://nerdculture.de/

PS: "A quick guide to The Free Network" by Sean Tilley on https://medium.com/we-distribute/a-quick-guide-to-the-free-network-c0693...

PPS: this is a cross post from my Friendica node.

Kategorie: DebianTags: DebianRubySocial NetworkPHPServer 
Kategorien: Private Blogs