Redmine › SCM Creator (+Github)

Download SCM Creator 0.5.0b

Own SCM server has many benefits such as the fully controlled access and security of repositories. But such services as Github, in most cases, provide more features. It’s explainable as many people work on features and ease of use on Github and you will unlikely be able to assign such amount of specialists just for your local SCM server. Therefore, many companies and individuals prefer to use Github for hosting their repositories...

The “virtual” SCM¶

However, you’re going to have troubles, if you want to use Github repositories with (pure) Redmine... While Redmine does supports Git (the backend SCM, which is used by Github) it does not support remote Git repositories! So, Github fans are forced to manually make local mirrors of their Github repositories especially for Redmine (as described here). This procedure, actually, looks similar to what SCM Creator automates for other SCMs, doesn’t it?.. So, I asked myself: Why would not we automate this too? That’s how SCM Creator got Github support!

SCM Creator 0.5.0b adds Github as a new SCM (while, actually, it is not new SCM – it’s Git, in fact). I call this new SCM “virtual”. It gets registered in the system at the same level as Git, Subversion and so on and is listed under the Administration → Settings → Repositories tab:

It looks similar to Git, because it’s, actually, Git (and that’s why I call it virtual)! Github virtual SCM is used by SCM Creator just to allow Git to make a local mirror of the remote repository. After that the local repository is used as just Git.

Additionally the Github SCM makes SCM Creator update the local mirror from the original remote repository, when the appropriate command is executed by Redmine (read details below)...

The Creator¶

This way SCM Creator adds another SCM to Redmine... But the main goal of the SCM Creator is different! Remember, we use it to create new repositories? While the local mirror is created for Github, it’s not actually a new repository. But, SCM Creator is also able to create new remote repositories on Github!..

When using Github for hosting your company’s repositories, you usualy have a guy, who is able to login into your company’s account on Github and who creates repositories, when someone requests them. The similar is done by SCM server administrators, if SCM Creator is not installed... That’s when SCM Creator actually helps! And now it does this for Github as well!..

Luckily, Github provides API for creating repositories and this API can be used by SCM Creator to make your life easier (or the life of that guy)! When properly configured SCM Creator adds the beloved “Create new repository” button for Github SCM. This button lets creating remote Github repositories with just one click! So, you only need to make sure, that the “Manage repository” permission is granted to the right guys (role) in your Redmine.

The commits fetching¶

However, not everything is smooth about Github... As it actually uses the remote repository you may experience performance issues. Similar issues happen, when remote Subversion repositories are used, so these are not new issues and are not specific to Github and, therefore, (luckily) the solution exists...

On the Repositories tab of Administration → Settings page you can see the “Fetch commits automatically” option, which is enabled by default. When this option is checked, Redmine updates the repository (or rereads for local one) each time you access any page under the Repository tab of the project. For remote Subversion and Github this means, that Redmine fetches latest commits from the remote repository, so certainly it’s appreciable.

Sometimes, this process is appreciable for local repositories as well, especially if the local repository is huge. This happens because Redmine also updates the internal cache (which is stored in the database).

So what is the solution?.. First, the “Fetch commits automatically” option must be disabled. Second, the “Enable WS for repository management” option must be enabled.

When the web service for repository management is enabled, Redmine updates repository commits (including the internal cache) only when this was explicitly requested! And this can be requested using the web service client, which is usually put into SCM repository hook or into the cron. For manually mirrored Github repositories (i.e., not what SCM Creator does) it’s also common to use the Github Hook plugin of Jakob Skjerning to update the mirror from the original remote repository. Anyway, configuring this is a quite complex procedure, that requires some manual work, similar to which SCM Creator aims to prevent... So, it prevents this for Github as well!

Github’s Git repository like any other Git repository also supports hooks. Of course, Github does not provide the direct access to repository system files (which is required to use hook scripts), but instead it provides the friendly interface for this. Additionally, it makes it possible to create hooks using Github API – the same API, which SCM Creator uses to create repositories. So, it’s quite possible for SCM Creator to create and configure Github hooks remotely without any manual work... And it does this!

When the “Register hook for fetching commits” option in the repository form is checked, Redmine also registers the Redmine service hook for the Github repository. The Github’s Redmine service hook was specially designed for Redmine to explicitly trigger the commits fetching, that we have discussed above. This way as soon as you enable this option you can be sure, that Redmine will update its internal commits cache right after new changes are committed to the Github repository. And this works for both: newly created Github repositories and already existing Github repositories! Moreover, this option remains available even for repositories, that have been already added to Redmine...

Certainly, to register the hook SCM Creator must have appropriate permissions for the Github repository. They are granted, if it creates new repository, but for existing ones it needs login and password for the appropriate Github account...

Download SCM Creator 0.4.3

Some time has passed since the last release of the SCM Creator and several new versions of Redmine has been released since then. So, certainly, preparing this release I was looking to making the plugin compatible to the recent Redmine mostly. Additionally I aimed to fix all known bugs... This way this version comes with support for Redmine 2.2.x and 2.3.x and with many bug fixes. Curiously, most of these bugs were related to Redmine 1.4.x, so this release can be considered to come with a better support for this Redmine version.

By the way, many guys helped me in fixing bugs for version 0.4.3... Thus, Lluís researched and fixed the attributes issue (#2135), Alexandre Lessard researched and fixed the array concatenation issue (#2137), Jun Naitoh fixed language string issues (#2183) and Yukinari Toyota fixed escaped HTML code issue (#2203). Thank you, guys!

Unfortunately not many new features can be found in 0.4.3 (what makes it a maintenance release mostly)... But one can be! Like for Extended Fields (as noted here) I decided to demonstrate, why voting and/or watching is important, by choosing an issue with the highest these numbers and implementing it for 0.4.3! This way the repository list now renders external URLs instead of local server paths, as it was requested in #2078.

This feature, in fact, was omitted as pre-1.4.x versions of the plugin (i.e., ones, which supported a single repository per project) did showed the external URL in the repository form (instead of the hint under the “URL” field). So, certainly, it should have been working this way from the beginning (thanks Humberto Anjos for drawing my attention to the issue).

To activate the feature you just need to specify url in your Redmine configuration file (and restart, of course):

production:
  svn:
    url: svn
  git:
    url: http://git.myhost.com

In addition, the version 0.4.3 includes an update to the Japanese translation, which was provided by Jun Naitoh. Thanks, Jun!

Not much time has passed since the release of 0.4.1 as new version of Redmine has come. This was Redmine 2.1.x, which (much similar to 2.0.x) switched to the new version of Rails – 3.2.x. Luckily, new Rails did not introduce so many changes as 3.0.x. However, it moved from Prototype to jQuery... So, all the JavaScript code needed to be fixed. As usually these fixes do not break the plugin’s compatibility with previous versions of Redmine down to 1.0.x.

While porting to Redmine 2.1.x was not a very complicated task, two guys really helped me to coomplete it by testing the plugin on the recent Redmine, by researching reasons of issues and by providing patches to the code. These guys are Martin Corino and Takashi Okamoto. Thank you, guys! It’s worth mentioning, that it’s not the first time, when Takashi Okamoto contributes to the SCM Creator code...

Martin Corino, however, helped not only with porting... He found the bug #2102 and provided the fix for it. I guess circumstances, which make Martin find this bug, were not pleasant. Sorry for this, Martin, and thanks for the fix!

Perhaps, I hurried, when was working on the 0.4.1, as this release included another bug – #2095, or its duplicate #2120. Both issues included the patch fixing the bug. These issues were reported and patches were provided by Manfred Kröhnert and Christian Rishøj. Thank you, guys!

Also with 0.4.2 the SCM Creator got the Korean translation. Thanks to Jaebok Oh!

If you want to be among the first, who gets notified about upcoming versions of SCM Creator, subsribe to this project using the Subscription block on the sidebar.

SCM Creator 0.4.0 was aimed to support Redmine 2.0.x and Rails 3... So to port it I installed clean Redmine 2.0.x with SCM Creator 0.3.0 and the first thing I noticed was that I could not migrate the plugin because of error:

uninitialized constant RepositoryObserver

So I applied the fix:

Rails.configuration.active_record.observers << :repository_observer

After that the plugin migrated! Being happy with it I continued porting...

Then, almost before making the release, I discovered that RepositoryObserver does not get called! And, to my surprise, returning to ActiveRecord::Base.observers << :repository_observer worked for Redmine 2.0! I assumed that there was something else, that I had fixed along with this issue, and made the release...

But then it has been discovered that it does not work for all of us! Fixing this weird issue has ended with this emergency release!

Many thanks to Rob Landry for discovering the issue and assisting in fixing it!

Thanks also to Ismail SEZEN for Turkish translation and to Fabien Crespel for update of French translation!

Redmine 1.4.x and Redmine 2.0.x were released quite a long time ago and only now SCM Creator supporting these versions was released! And it is only the first my plugin for 2.0.x...

This release could not, perhaps, come so “early” without the kick of Takashi Okamoto! He assisted me a lot in porting SCM Creator to multiple repositories and Rails 3/Redmine 2. He even created a fork for doing this on Github: https://github.com/okamototk/redmine_scm_creator. Thanks Takashi Okamoto!

Before I planned to release it as 0.3.1. But thanks to Okamoto who suggested 0.4.0 I changed my mind. It’s too much changes for versioning it as 0.3.1. So it is 0.4.0!

So let’s talk about changes:

Support of multiple repositories, which appeared in Redmine 1.4.x, was not a feature for me... it was a headache!

Before there was a single repository and one, who was willing fo have all repositories created, could just set auto_create to force and deny_delete to true. This one could be sure that no one was going to add external repositories. Now this got broken!.. So as a workaround I added only_creator option which allows only creating repositories and denies adding, however SCMs not supported by this plugin should be disabled...

But that was not the main issue! How many repositories can one create?.. So I thought it’s not a good idea to let users creating infinite number of repositories. That’s why I also added max_repos option. This option specifies the limit of repositories which can be created using SCM Creator.

A long time ago Blaž Podržaj discovered an obvious issue which I missed for some reason... When auto_create was set a user was suggested (or forced) to create a repository on project creation. And it got created despite the value of the “Repository” checkbox under modules list... With this release SCM Creator takes this value into account and won’t create repository if it is not checked! But...

What should those users do who liked this behaviour?.. Before they were getting repository despite of the “Repository” module availability. I can imagine that some users reckon on the old behaviour... So for such users I added force_repository option which just enables “Repository” module and does not let disabling it!

Note that it is still possible to disable the “Repository” module later in project settings! It’s not a bug - it’s current implementation as I want to see what users think about the force_repository option... And by disabling “Repository” users will just hide it, not remove.

Also all the time in Redmine there was a security issue... Redmine guys name it “invalid configuration issue” and recommend giving the :repository_manage permission only to trusted users. I don’t agree. You can trust users but can you trust their passwords? Can you trust their Internet connection?.. With multiple repositories this issue became even more noticable so I decided to provide and enforce the solution. This solution uses two new options: allow_add_local and allow_pickup.

This release of SCM Creator has also a special meaning for my other plugins - having practiced with porting Redmine plugins to Rails 3/Redmine 2, I’m sure, I will port other plugins faster (unless they contain some bugs)...

Also reminding that you can subscribe to news, new versions, commits etc by using Subscription block on the sidebar.

P.S. WikiNG, you come next!

From release to release the SCM creator becomes more powerful! Now it’s not only my personal plugin which was just shared... It has been extended mostly by community requests and patches. This release is not an exclusion - new breaking features were suggested or supported by Mario Luzeiro, Blaž Podržaj and Jonas Götze (thanks, guys!).

Traditionally new major version is released with the support of new SCM - this time Bazaar. Adding Bazaar support (and help in testing) was suggested by Mario Luzeiro! So without him this release would not happen!

To add Bazaar support I also did what I wanted to do a lot time ago: finally I did refactored the plugin to make adding new SCMs easier! This also made the code more clear and I hope more bug-proof.

Another breaking feature was suggested by Blaž Podržaj. Thanks to him new release allows specifying on creation and on deletion scripts. These scripts can be used for preparing repositories for usage and archiving repositories before deleting. By the way, thanks also to Jonas Götze for help in improving scripts!

These scripts can do the same what hooks options introduced in the previous version do... Therefore hooks now should be treated obsolete and one using it should switch to on creation script (check this page how)! In one of further versions the hooks support may be removed...

The last feature was needed by me but you can find it useful too if you use my Project Alias plugin. New version is able to rename repository when project identifier is renamed. Please note that this will also change repository access URL (old repository URL will stop working)! That is be sure to notify users before the change!

Also in this version other smaller improvements and bug fixes were made...

That seems to be all!.. Write me, suggest me new features! Let’s make this plugin better!

And thanks for using it!

As I had mentioned some time ago I changed the project identifier from the obsolete svn to more correct and SEO friendly scm-creator! Thanks to my another plugin old URLs will still work because the svn identifier is now an alias. I have also added the alias redmine-scm (similar to the plugin’s internal ID)...

Unfortunately this does not apply to SVN URLs - old SVN links stopped to work! Sorry for this!

New SVN URL is: http://svn.s-andy.com/scm-creator

To update the URL in your SVN working copy do:

$ svn switch --relocate http://svn.s-andy.com/redmine-svn http://svn.s-andy.com/scm-creator

Where,

• http://svn.s-andy.com/redmine-svn - old URL
• http://svn.s-andy.com/scm-creator - new URL

This change also precedes new SCM Creator release!.. So be ready for updates!

Like the version 0.1.0 version 0.2.0 comes with new SCM support - it’s now possible to create local Mercurial repository with just one click. This version also comes with hooks support for all currently supported SCMs - you can now specify a directory/file containing hook scripts and they will be automatically copied into a newly created repository.

The version 0.2.0 also introduces new features for Git: now it supports new option update-server-info which is required for Redmine/Git integration (to let Git work through HTTP) and new option git_ext which lets enabling/disabling .git extension.

And perhaps the most wanted change is support for Redmine 1.2.x and ChiliProject!

But there is also one thing you should note: in previous versions you could press on the “Delete” button under “Repository” tab in project settings and be sure that local repository created with the SCM Creator remains on the file system. You could even get used to this... It’s not so anymore! Now when you delete a repository created with the SCM Creator (e.g. remove project) it is also removed from the file system!

This release would not be possible without help of: John Blackwood and Etienne Savard (Mercurial), Takashi Okamoto (hooks patch), Paramvir Kaler (Git testing), Roland Firmont (Redmine 1.2.x), Fabricio Caseiro and Txinto Vaz (translations). Thank you, guys!

Here it goes - the SCM Creator 0.1.0 is released!

• Git support
• Windows support
• Automatic creation of repositories on projects registration.

For full list check: Changes in 0.1.0.

This version (as it is) would not be released without contribution of: Bertrand Baudinet (initial Git patch), Varun Shoor (idea of automatic creation), Jean-Sebastien Bour (many patches and ideas) and Andrey Reeshkov (Windows testing)! Thank you, guys!

Right now I’m working on adding full Git support to the plugin (#1707). Besides the new version is going to support Windows (#1692) at least for Subversion (by the way, I need help in testing Windows support for both - SVN and Git).

So too much changes to preserve the current name and I decided to change it to “SCM Creator”. In future I plan to add support for other SCMs as well...

P.S. Besides the Git and Windows and name change the upcoming release will have some new interesting features, for example, automatic repository creation on project registration (many thanks to Varun Shoor for the idea)!