Scrutinizer Migration Guide

Scrutinizer is an enterprise/business class NetFlow and sFlow analysis tool. Scrutinizer provides historical trends of the company's critical network interfaces as well as the details on:

Who: The end system causing the traffic
What: The application/protocol that is being used
When: The time frame it has been occurring for
Where: The network connection that is affected

Moderators: scottr, Moderator Team

Post Reply
jennm
Posts: 6
Joined: Thu Jun 09, 2016 11:36 am

Scrutinizer Migration Guide

Post by jennm » Thu Aug 04, 2016 10:39 am

Plixer Scrutinizer DB Migration Utility


DOWNLOAD

https://files.plixer.com/support/migrate.tar.gz

SYNOPSIS
OPTIONS...
migrate.exe [-d|-f|-m|-p|-r|-t]

Migrates a Scrutinizer install between different hosts, OS, and DB platforms. Supported Database platforms are !MySQL and !PostgreSQL. **Please be sure you have READ and completely UNDERSTAND the documentation before using the util on any kind of production database.**

Requires one of the run mode flags:

Run Modes:

Code: Select all

 -d, -data         Migrate Scrutinizer historical data.  You MUST be collecting
                   flows on the destination before doing this.
 -f, -finish       Attempt to remove DB ROLEs listed in the migrate.ini and
                   outputs necessary SQL commands to do so if not.
 -m, -meta         Migrate Scrutinizer meta data.  The plixer_flow_collector
                   service MUST be stopped on the destination.
 -p, -push         Migrate a single, specific table.
                   ex: ./migrate.exe -p plixer.ip_groups
 -r, -retreat      'Un-migrates' historical data at the destination server.
 -t, -test         Tests connections and gathers some initial migration stats.

Options:

Code: Select all

 -c, -cross_ver,   Used for cross-version meta data migrations.  Stomps SCHEMA
 -clobber_version  on the destination and re-CREATEs it using the source as
                   reference.  You MUST re-run the destination's Scrutinizer
                   installer after the meta data migration.   
 -checksum         Performs a checksum per table for migrated historical data 
                   and logs the results. 
 -dump             Performs a mysqldump or pg_dump of the Scrutinizer meta
                   data pre-migration to scrutinizer/files.
                   Providing you have either mysqldump or pg_dump installed and
                   in your environment PATH.
 -flip_ips         Will "flip" the source and destination ips in the
                   migrate.ini.  Handy if you're re-IPing the destination
 -h, -help         Display this help message.
 -i, -inifile      Designate a path to a configuration file. 
                   scrutinizer/files/migrate.ini by default.
 -l, -logfile      Designate the path to write a migration log to. 
                   scrutinizer/files/logs/migrate.log by default. 
 -man, -info       Display the full documentation.                    
 -q, -quiet        Suppress terminal progress messages. 
 -skip_clock       Skips source/destination clock test (as risky as it sounds)
 -skip_lock        Skips table locking during migration (as risky as it sounds)

DESCRIPTION

The Plixer Scrutinizer DB Migration Utility is designed to ease the process of moving a Scrutinizer install between hosts and/or Database platforms. As with any kind of critical database work, a complete back-up should be made before attempting any kind of migration.


PRE-MIGRATION

The Migration Utility requires database ROLEs with SUPERUSER privileges that can connect remotely. Since this is an obvious security risk, temporary ROLEs should be created for the duration of the migration.

These Examples assume 'tmp_user' and 'tmp_pass' as the ROLE.

MySQL

Code: Select all

  SELECT PASSWORD('tmp_pass');
  +-------------------------------------------+
  | PASSWORD('tmp_pass')                      |
  +-------------------------------------------+
  | *00C233A85F04B58B5C9728B5BB411DC46F85BA54 |
  +-------------------------------------------+
  GRANT ALL ON *.* TO 'tmp_user'@'%' IDENTIFIED BY PASSWORD '*00C233A85F04B58B5C9728B5BB411DC46F85BA54'; 
  FLUSH PRIVILEGES;

PostgreSQL

Code: Select all

CREATE USER tmp_user WITH SUPERUSER ENCRYPTED PASSWORD 'tmp_pass';
You'll also need to update /home/plixer/scrutinizer/pgsql/data/pg_hba.conf to allow 'tmp_user' to connect remotely. Add this line to pg_hba.conf:

Code: Select all

host  all tmp_user  0.0.0.0/0 md5
Restart the plixer_db service

Warning

Be sure to remove these ROLEs once the migration is complete. the -finish run mode will help you with this.



CONFIGURATION

Most configuration parameters are set in the migration ini file, 'migrate.ini' by default. The Utility will look for this file in the present working directory if -inifile is not set.

[source|destination database]
DB connection parameters for the migration hosts.

==== *_db_host

IP address of the migration host. Use the loopback address if you're running this util locally on one of the hosts.

==== *_db_user

You will need to ensure that the db user named here has full GRANTs to the plixer SCHEMA as well as information_schema. 'root' is likely your best bet.

==== *_db_pass

Password for the Database User.

==== *_db_port

Database connection port. Default is 3306 for !MySQL , 5432 for !PostgreSQL.

==== *_db_dialect

This is the Database connection driver. 'mysql' for a !MySQL database. 'pgsql' for a !PostgreSQL database.

==== *_db

The database to use upon connection. Leave this blank for !MySQL. Default for !PostgreSQL is 'plixer'.


[exporters]


=== inclusions ===

A space separated list of exporters to include when performing a historical data migration with the -d or --data flag. ex: inclusions= 10.1.1.3 10.1.1.4 The migrate utility will attempt to migrate all exporters if left blank


=== exclusions ===

A space separated list of exporters to exclude when performing a historical data migration with the -d or --data flag. ex: exclusions= 6.6.6.6 7.7.7.7 The migrate utility will attempt to migrate all exporters if left blank


[data range]


=== intervals ===

A space separated list of flow history intervals to migrate. Valid entries are: 1m 5m 30m 2h 12h 1d 1w By default, 1 minute data is not migrated.


=== start/end_epoch ===

Limits the time frame of data migrations to a unix timestamp.


[scrutinizer data]

These are specific "categories" of Scrutinizer meta data you wish to migrate from the source install to the destination install. This is done when calling the utility with the -m or --meta flag.

Any value for these parameters in the migrate.ini is considered true. To not migrate a particular category, leave the value blank.


=== alarms ===

Migrates Alarm Bulletin Boards, Alarm Policies, Notification Policies, and Flow Analytics Configuration.


=== dashboards ===

Migrates all Dashboards and Dashboard Gadgets.


=== ipgroups ===

Migrates custom Applications and !IpGroups.


=== objects ===

Migrates Scrutinizer Maps and Device Tree Groups. If you're using any custom map background images, they will need to be uploaded manually.


=== reports ===

Migrates Saved Reports and any reports created in Report Designer.


=== systemprefs ===

Migrates Scrutinizer server preferences, scheduled tasks, Cross Check configuration, and any Third Party configuration.


=== users ===

Migrates Scrutinizer users, user groups, and user preferences.



EXAMPLES


== Scrutinizer Cross Version Migration ==

These steps work for either !MySQL or !PostgreSQL. The source server MUST be version 15.8 or greater.
  1. Deploy and license the latest Scrutinizer VA for the destination.
  2. Following the PRE-MIGRATION section, create temporary database ROLEs.
  3. Get the migration utility binary and migrate.ini file onto the destination.
  4. Following the CONFIGURATION section, fill out the migrate.ini.
  5. Stop the plixer_flow_collector service on the destination.
  6. Test the connections. Only proceed if you get all green [OK] results.

    Code: Select all

      migrate.exe -t
  7. Migrate the Scrutinizer meta data using the clobber_version flag.

    Code: Select all

      migrate.exe -c -m
  8. Run the latest Scrutinizer installer on the destination.
  9. Verify the meta data migration was successful via the web interface.
  10. Verify collection has started again. You MUST be collecting on the destination before continuing.
  11. Migrate the Scrutinizer historical data within tmux.

    Code: Select all

      tmux
      migrate.exe -d --skip_version
    
  12. Once the migration is complete, be sure to remove the temporary ROLEs.

    Code: Select all

      migrate.exe --finish
If you have any questions regarding the use of this utility please contact support: https://www.plixer.com/Support/support-overview.html

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest