Eyes, JAPAN Blog > Small tip for migrating from Trac to Redmine

Small tip for migrating from Trac to Redmine

denvazh

この記事は1年以上前に書かれたもので、内容が古い可能性がありますのでご注意ください。

How come?

We were using Trac for only one project in the company. (*Unfortunately) After sometime it was necessary to migrate all the project contents to the Redmine system. (*Fortunately) Redmine provides nice migration script called migrate_from_trac, thus it is very easy to migrate. (*Unfortunately, again) What seems easy, however, might be a bit harder when one would actually try using it, but I will return to it later.

Migrating

Redmine uses Rails, thus it is necessary to setup database and add connection information to the config/database.yml.
If is also required to install necessary database connectors for both Redmine and Trac databases. For example, Trac uses sqlite ( by default ),
so if it necessary to migrate from sqlite database, then one can install necessary connectors with:

$: gem install sqlite-ruby
or
$: gem install sqlite3-ruby

Finally, to migrate project it is necessary to run rake script with migrate_from_trac option. To migrate settings to specific database environment use RAILS_ENV environment variable. For example, to migrate to production database:

$: rake redmine:migrate_from_trac RAILS_ENV="production"
Trac directory []: /home/projects/myproject
Trac database adapter (sqlite, sqlite3, mysql, postgresql) [sqlite]:
Database encoding [UTF-8]:
Target project identifier []: myproject

Deleting data
Migrating components..............................
Migrating milestones..............
Migrating custom fields.......
Migrating tickets.................................
Migrating wiki...........

Components: 10/11
Milestones: 25/25
Tickets: 131/131
Ticket files: 203/203
Custom values: 15/15
Wiki edits: 1022/1022

Troubles?

In my case, I had trouble when migration scripts was unable to import entries with not correct timestamp.

$: rake generate_session_store
$: rake redmine:load_default_data RAILS_ENV="production"
$: rake redmine:migrate_from_trac RAILS_ENV="production"

WARNING: a new project will be added to Redmine during this process.
Are you sure you want to continue ? [y/N] y

Trac directory []: /home/projects/myproject
Trac database adapter (sqlite, sqlite3, mysql, postgresql) [sqlite]: sqlite3
Trac database encoding [UTF-8]: 
Target project identifier []: myproject

Migrating components....
Migrating milestones......
Migrating custom fields...
Migrating tickets.............................
Migrating wiki.rake aborted!
PGError: ERROR:  timestamp out of range: "41334730-01-26 05:38:55.492266"
LINE 1: ...otected", "parent_id", "wiki_id", "title") VALUES('41334730-...
                                                             ^
: INSERT INTO "wiki_pages" ("created_on", "protected", "parent_id", "wiki_id", "title") VALUES('41334730-01-26 05:38:55.492266', 'f', NULL, 2, 'DissGlossar') RETURNING "id"

Tasks: TOP => redmine:migrate_from_trac
(See full trace by running task with --trace)

After digging for a bit more time ( for instance running with suggested option –trace ) I noticed, that timestamp entries in sqlite database was stored as 16-digit long value, whereas postgres expected 10-digit long timestamp. Fixing content in the sqlite database is quite a hassle, because it involves changes in all entries for tickets and wiki, thus I created a patch to improve migrate_from_trac rake script in way, that every table that had timestamp field would be fixed during migration. This way data would be extracted from sqlite database as it is, then if timestamp value is 16-digit long, then it will reduce it to 10-digit ( basically drop last 6 zeros ) format.

To use this patch

  1. Copy and paste code below to the file migrate_from_trac.patch
  2. Put file to redmine project directory in “lib/tasks”
  3. From redmine root run command: cd lib/tasks && patch < migrate_from_trac.patch
  4. After patch was applied, run migration script again

Patch that allows to workaround migration of 16-digit format timestamp to database, that expects 10-digit long format:

--- migrate_from_trac.rake      2012-03-26 11:11:52.000000000 +0900
+++ migrate_from_trac.rake      2012-03-26 11:56:15.000000000 +0900
@@ -22,6 +22,27 @@
 namespace :redmine do
   desc 'Trac migration script'
   task :migrate_from_trac => :environment do
+    
+    module FixTime
+        # Workaround for import from sqlite3
+        # Fixing issue with timestamp length ( Trac - 16, Redmine - 10)
+        class TrueTime
+               attr_accessor :timestamp
+        
+               def initialize(input_timestamp)
+                       @timestamp = fix_timestamp(input_timestamp)
+               end
+        
+               def fix_timestamp(timestamp)
+                       if (timestamp && timestamp > 9999999999)
+                               new_time = timestamp/1000000
+                               return new_time.ceil
+                       else
+                               return timestamp
+                       end
+               end # fix_timestamp
+        end # class
+    end
 
     module TracMigrate
         TICKET_MAP = []
@@ -69,6 +90,7 @@
                         'developer' => developer_role
                         }
 
+
       class ::Time
         class << self
           alias :real_now :now
@@ -93,7 +115,9 @@
         # If this attribute is set a milestone has a defined target timepoint
         def due
           if read_attribute(:due) && read_attribute(:due) > 0
-            Time.at(read_attribute(:due)).to_date
+               duetime = FixTime::TrueTime.new(read_attribute(:due))
+               due = duetime.timestamp
+            Time.at(due).to_date
           else
             nil
           end
@@ -101,7 +125,9 @@
         # This is the real timepoint at which the milestone has finished.
         def completed
           if read_attribute(:completed) && read_attribute(:completed) > 0
-            Time.at(read_attribute(:completed)).to_date
+               completed = FixTime::TrueTime.new(read_attribute(:completed))
+               completed = completed.timestamp
+            Time.at(completed).to_date
           else
             nil
           end
@@ -121,7 +147,10 @@
         set_table_name :attachment
         set_inheritance_column :none
 
-        def time; Time.at(read_attribute(:time)) end
+        def time
+         time = FixTime::TrueTime.new(read_attribute(:time))
+          Time.at(time.timestamp)
+        end
 
         def original_filename
           filename
@@ -182,14 +211,24 @@
           read_attribute(:description).blank? ? summary : read_attribute(:description)
         end
 
-        def time; Time.at(read_attribute(:time)) end
-        def changetime; Time.at(read_attribute(:changetime)) end
+        def time
+         time = FixTime::TrueTime.new(read_attribute(:time))
+          Time.at(time.timestamp)
+        end
+
+        def changetime
+         time = FixTime::TrueTime.new(read_attribute(:changetime))
+          Time.at(time.timestamp)
+        end
       end
 
       class TracTicketChange < ActiveRecord::Base
         set_table_name :ticket_change
 
-        def time; Time.at(read_attribute(:time)) end
+        def time
+         time = FixTime::TrueTime.new(read_attribute(:time))
+          Time.at(time.timestamp)
+        end
       end
 
       TRAC_WIKI_PAGES = %w(InterMapTxt InterTrac InterWiki RecentChanges SandBox TracAccessibility TracAdmin TracBackup TracBrowser TracCgi TracChangeset \
@@ -214,7 +253,10 @@
           super.select {|column| column.name.to_s != 'readonly'}
         end
 
-        def time; Time.at(read_attribute(:time)) end
+        def time
+         time = FixTime::TrueTime.new(read_attribute(:time))
+          Time.at(time.timestamp)
+        end
       end
 
       class TracPermission < ActiveRecord::Base

Now migration should go without and further problems. Let’s try again.

$: rake redmine:migrate_from_trac RAILS_ENV="production"

WARNING: a new project will be added to Redmine during this process.
Are you sure you want to continue ? [y/N] y

Trac directory []: /home/projects/myproject
Trac database adapter (sqlite, sqlite3, mysql, postgresql) [sqlite]: sqlite3
Trac database encoding [UTF-8]: 
Target project identifier []: myproject

Migrating components....
Migrating milestones......
Migrating custom fields...
Migrating tickets.............................
Migrating wiki..........................................................................................................................................................................................................................................................................................................................

Components:      0/4
Milestones:      0/6
Tickets:         29/29
Ticket files:    1/1
Custom values:   84/84
Wiki edits:      314/314
Wiki files:      21/22

One response to “Small tip for migrating from Trac to Redmine”

  1. Domen Kožar says:

    I solved it by converting all time columns like:

    UPDATE ticket SET time = time / 1000000.0 where time > 9999999999;