Skip to content

u2g mmi

Jump to bottom
mmisw edited this page Jan 13, 2014 · 35 revisions

System

$ cat /etc/*release
CentOS release 5.9 (Final)

Verify and enable base software

Apache HTTP server

$ /sbin/chkconfig --list httpd
httpd          	0:off	1:off	2:off	3:off	4:off	5:off	6:off

$ sudo /sbin/chkconfig --level 235 httpd on
$ /sbin/chkconfig --list httpd
httpd          	0:off	1:off	2:on	3:on	4:off	5:on	6:off

$ sudo /sbin/service httpd start
Starting httpd:                                            [  OK  ]

$ links http://localhost  #  OK

External access to http://marinemetadata.org/ OK (default apache 2 test page).

MySQL

$ /sbin/chkconfig --list mysqld
mysqld         	0:off	1:off	2:off	3:off	4:off	5:off	6:off
$ sudo /sbin/chkconfig --level 235 mysqld on
$ /sbin/chkconfig --list mysqld
mysqld         	0:off	1:off	2:on	3:on	4:off	5:on	6:off

$ sudo /sbin/service mysqld  start
Initializing MySQL database:  Installing MySQL system tables...
...
Starting mysqld:                                           [  OK  ]


$ sudo /usr/bin/mysql_secure_installation
[sudo] password for wwwww:




NOTE: RUNNING ALL PARTS OF THIS SCRIPT IS RECOMMENDED FOR ALL MySQL
      SERVERS IN PRODUCTION USE!  PLEASE READ EACH STEP CAREFULLY!


In order to log into MySQL to secure it, we'll need the current
password for the root user.  If you've just installed MySQL, and
you haven't set the root password yet, the password will be blank,
so you should just press enter here.

Enter current password for root (enter for none):
OK, successfully used password, moving on...

Setting the root password ensures that nobody can log into the MySQL
root user without the proper authorisation.

Set root password? [Y/n] Y
New password:
Re-enter new password:
Password updated successfully!
Reloading privilege tables..
 ... Success!


By default, a MySQL installation has an anonymous user, allowing anyone
to log into MySQL without having to have a user account created for
them.  This is intended only for testing, and to make the installation
go a bit smoother.  You should remove them before moving into a
production environment.

Remove anonymous users? [Y/n] Y
 ... Success!

Normally, root should only be allowed to connect from 'localhost'.  This
ensures that someone cannot guess at the root password from the network.

Disallow root login remotely? [Y/n] Y
 ... Success!

By default, MySQL comes with a database named 'test' that anyone can
access.  This is also intended only for testing, and should be removed
before moving into a production environment.

Remove test database and access to it? [Y/n]
 - Dropping test database...
 ... Success!
 - Removing privileges on test database...
 ... Success!

Reloading the privilege tables will ensure that all changes made so far
will take effect immediately.

Reload privilege tables now? [Y/n]
 ... Success!

Cleaning up...



All done!  If you've completed all of the above steps, your MySQL
installation should now be secure.

Thanks for using MySQL!

Base location

$ sudo mkdir /var/lib/marinemetadata/
$ cd /var/lib/marinemetadata/
$ sudo chown `whoami` .
$ df -h .
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_internal-lv_root
                      138G  2.6G  129G   2% /

Drupal contents and database on previous host

Drupal site contents:

$ cd /var/lib/marinemetadata/
$ du -sh *
33G	from_mbari
3.6G	mmi
4.3G	svnrepos

$ mkdir to_tamucc
$ nohup tar zcf to_tamucc/mmi-drupal-site-2013-12-03.tgz mmi &

$ sftp [email protected]
sftp> put mmi-drupal-site-2013-09-30.tgz
sftp> ls -l
-rw-r--r--    1 1053     1018     3442997329 Oct  1 03:23 mmi-drupal-site-2013-09-30.tgz

Database:

$ mysqldump -u root -p drupal_mmi > drupal_mmi.dump-2013-09-30.sql
Enter password:

Note: initially, this generated:

mysqldump: Got error: 1449: The user specified as a definer ('drupal_mmi'@'localhost') does not exist when using LOCK TABLES
$ mysql -u root -p
Enter password:
Server version: 5.5.17 MySQL Community Server (GPL)
mysql> create user 'drupal_mmi'@'localhost' identified by 'xxxxxx';
Query OK, 0 rows affected (0.04 sec)
mysql> GRANT ALL PRIVILEGES ON *.* TO 'drupal_mmi'@'localhost';
Query OK, 0 rows affected (0.01 sec)

$ nohup mysqldump -u root -p drupal_mmi > to_tamucc/drupal_mmi.dump-2013-12-03.sql
$ cd to_tamucc/
$ l -h drupal_mmi.dump-2013-12-03.sql
-rw-r--r-- 1 carueda 1068 24G Sep 30 20:03 drupal_mmi.dump-2013-09-30.sql
$ gzip drupal_mmi.dump-2013-12-03.sql
$ sftp [email protected]
sftp> put drupal_mmi.dump-2013-12-03.sql.gz
sftp> ls -l

Drupal contents and database on new host

Transfer files from sftp space:

$ cd /var/lib/marinemetadata/
$ mkdir from_ucsd
$ cd from_ucsd 
$ sftp [email protected]
sftp> ls -l
-rw-r--r--    1 uuuu   MMI-sftp-users 415893754 Oct  1 03:25 drupal_mmi.dump-2013-09-30.sql.gz
-rw-r--r--    1 uuuu   MMI-sftp-users 3442997329 Oct  1 03:23 mmi-drupal-site-2013-09-30.tgz
sftp> get mmi-drupal-site-2013-09-30.tgz
sftp> get drupal_mmi.dump-2013-09-30.sql.gz

$ ls -lrt
total 3772152
-rw-r--r-- 1 wwwww wwwww 3442997329 Oct 19 14:04 mmi-drupal-site-2013-09-30.tgz
-rw-r--r-- 1 wwwww wwwww  415893754 Oct 19 14:05 drupal_mmi.dump-2013-09-30.sql.gz

Drupal site contents:

$ cd /var/lib/marinemetadata/
$ tar zxf from_ucsd/mmi-drupal-site-2013-09-30.tgz
$ du -sh *
3.6G	from_ucsd
3.6G	mmi

$ df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/mapper/vg_internal-lv_root
                      138G  9.7G  121G   8% /
/dev/sda1              99M   20M   75M  21% /boot
tmpfs                 3.0G     0  3.0G   0% /dev/shm

Fix file ownership/permissions:

The following according to http://drupal.org/node/244924, except that we are not allowing file uploads through the site:

$ cd /var/lib/marinemetadata/mmi/
$ sudo chgrp -R apache .
$ sudo find . -type d -exec chmod u=rwx,g=rx,o= {} \;
$ sudo find . -type f -exec chmod u=rw,g=r,o= {} \;

Adjust settings

$ vi sites/default/settings.php   # to adjust $db_url = 'mysql://uuuu:pppp@localhost/drupal_mmi';

Database

$ cd /var/lib/marinemetadata/from_ucsd/
$ gunzip drupal_mmi.dump-2013-09-30.sql.gz

$ mysql -u root -p
mysql> create database drupal_mmi;
Query OK, 1 row affected (0.01 sec)

$ mysql -u root -p drupal_mmi < drupal_mmi.dump-2013-09-30.sql

mysql> SELECT table_schema "DB Name", Round(Sum(data_length + index_length) / 1024 / 1024, 1) "DB Size in MB" FROM   information_schema.tables GROUP  BY table_schema;
+--------------------+---------------+
| DB Name            | DB Size in MB |
+--------------------+---------------+
| drupal_mmi         |       22016.7 |
| information_schema |           0.0 |
| mysql              |           0.5 |
+--------------------+---------------+
3 rows in set, 1 warning (0.31 sec)

httpd.conf

$ cd /etc/httpd/conf
$ sudo cp -p httpd.conf httpd.conf_original
$ sudo vi httpd.conf # settings to dispatch the drupal site at /var/lib/marinemetadata/mmi

Initial tests

Mailman

Refs:

$ sudo yum install mailman
...
Installed:
  mailman.x86_64 3:2.1.9-6.el5_6.1

$ cd /usr/lib/mailman/
$ bin/version
Using Mailman version: 2.1.9

$ sudo /sbin/chkconfig --level 2345 mailman on
$ sudo /sbin/chkconfig --list mailman
mailman        	0:off	1:off	2:on	3:on	4:on	5:on	6:off

On previous host

$ cd /var/lib/mailman/
$ sudo tar zcf  /var/lib/marinemetadata/to_tamucc/mailman-files-2013-12-03.tgz lists data archives

$ cd /var/lib/marinemetadata/to_tamucc/
$ ls -l mailman-files-2013-12-03.tgz
-rw-r--r-- 1 uuuu 1068 ??? ??? mailman-files-2013-12-03.tgz
$ sftp [email protected]
sftp> put mailman-files-2013-12-03.tgz

On new host

$ cd /var/lib/marinemetadata/from_ucsd
$ sftp ... > get mailman-files-2013-12-03.tgz

$ cd /var/lib/mailman/
$ ls -lrt
total 32
drwxrwsr-x 2 root mailman 4096 Mar 30  2011 spam
drwxrwsr-x 2 root mailman 4096 Mar 30  2011 lists
drwxrwsr-x 2 root mailman 4096 Oct 19 19:24 data
drwxrwsr-x 4 root mailman 4096 Oct 19 19:24 archives
$ sudo tar zcf original_lists_data_archives.tgz lists data archives

$ sudo tar zxf /var/lib/marinemetadata/from_ucsd/mailman-files-2013-12-03.tgz

# cd /usr/lib/mailman/bin/
# ./check_perms -f 
Warning: Private archive directory is other-executable (o+x).
         This could allow other users on your system to read private archives.
         If you're on a shared multiuser system, you should consult the
         installation manual on how to fix this.
No problems found

# chmod o-x /var/lib/mailman/archives/private

# ./check_perms -f
No problems found

Regenerate aliases and "fix" URLs:

# cd /usr/lib/mailman/bin/
# ./genaliases > /tmp/mailman_genaliases_output
# vi /etc/aliases  # to append the aliases captured in /tmp/mailman_genaliases_output

# newaliases
/etc/aliases: line 271: news... Warning: duplicate alias name news
/etc/aliases: 556 aliases, longest 63 bytes, 30949 bytes total

# vi /etc/aliases  # to comment out the 'news' entry from mailman

# newaliases
/etc/aliases: 546 aliases, longest 63 bytes, 30396 bytes total

# for list in `ls /var/lib/mailman/archives/private/ | grep -v "\\.mbox"`; do \
        echo $list; ./withlist -l -r fix_url $list -u marinemetadata.org; done

Note: using marinemetadata.org even though cutover not in place yet.

@marinemetadata.org regular email aliases

To support the usual accounts @marinemetadata.org used in several mailing lists as administrators/moderators also the following lines were added to /etc/aliases:

techlead:       carueda@____
graybeal:       jbgraybeal@____
palexander:     paul@____

mailman.conf

# vi /etc/httpd/conf.d/mailman.conf # to indicate: RedirectMatch ^/mailman[/]*$ http://marinemetadata.org/mailman/listinfo

#  /etc/init.d/mailman start
Starting mailman:                                          [  OK  ]

# /usr/sbin/apachectl graceful

Reset site password:

# cd /usr/lib/mailman/bin/
# ./mmsitepass
New site password:
Again to confirm password:
Password changed.

Test: http://marinemetadata.org/mailman shows a "welcome" page with the mailing lists.

Access to the list archives (eg. http://marinemetadata.org/pipermail/ontdev/) showed "forbidden" errors, which I fixed by:

# cd /var/lib/mailman/archives
# chown apache private

mail tests

from same local machine:

echo "From: [email protected]
To: [email protected]
Subject: test from local
message sent from the same machine as a test for delivery.
" | /usr/sbin/sendmail [email protected] [email protected]

Successful: http://marinemetadata.org/pipermail/mailman/2013-October/000014.html

from outside:

Also working.

Log

2014-01-13

  • Pending transfer completed: SVN repository dump from_mbari/svnrepos-mmiweb-2012-05-01-1220.dump --> from_ucsd/svnrepos-mmiweb-2012-05-01-1220.dump. Note, it's the dump from the original host as we haven't used the repo since the migration from there.

2013-12-19

  • on new host:
    • enabled crontab for drupal's cron.php (with @midnight, so, not as frequently as before, given the site is not as active).
  • on old server (while it gets shutdown):
    • disabled crontab for drupal's cron.php
    • disabled mailman (/sbin/chkconfig --level 235 mailman off; /sbin/service mailman stop)
  • Noticed that admin changes in the lists using the web interface were not taking effect (btw, without any warning or error message). The behavior is particularly related with the default HTTPS access. The fix (see this):
    • edit /etc/mailman/mm_cfg.py to add DEFAULT_URL_PATTERN = 'https://%s/mailman/'
    • re-run the "fix_url" step described above.

2013-12-18

  • DNS switch applied at around 1:30pm pacific (thanks Adam).
  • Added a line to site footer ("This site is hosted by the Harte Research Institute, Texas A&M University - Corpus Christi, Corpus Christi, TX, USA")

2013-12-16/17

  • final file transfer and import (fresh drupal DB dump, and drupal/mailman files from Dec 3 -- no changes have happened since then so we should be good).
  • add "AllowOverride All" to /etc/httpd/conf.d/ssl.conf so clean-urls work with https access.
  • re-enable http->https redirection.
  • redo chown apache /var/lib/mailman/archives/private to avoid forbidden errors accessing list archives
  • test of external email to techlead@__ get delivered: OK
  • test of local email to mailman@__ get delivered and archived: OK

2013-12-03

  • dump of files on previous host (drupal files, drupal db, mailman files)

2013-10-21

  • mail port open by IT; mail lists seem to be working in general.
  • adjustments in drupal site and mailman to reflect the actual final domain name marinemetadata.org. Tests done via temporary adjustment to /etc/hosts.

2013-10-19

  • re-did settings for drupal site and database on reinstalled server.
  • initial tests for drupal site now ok for http; some issues with https
  • initial mailman setup and migration of lists

2013-10-02

2013-09-30

  • drupal site tarball created and transferred to tamucc's ftp server ("sftp tamucc" connection was failing initially)
  • mysql db (drupal_mmi) dump created and transferred to tamucc's ftp server.

2013-09-22

  • reviewed server status at new hosting and did basic installations
Clone this wiki locally