Author Topic: Building git repo  (Read 13577 times)

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Building git repo
« on: November 30, 2013, 11:45:55 pm »
Hi,
every time i build c::b from a git repo (https://github.com/obfuscated/codeblocks_sf/tree/no_ui_sdk) i get
Code
svn: E155007: '[...]\src' is not a working copy
and autorevision starts a huge tree of git/perl executables. I don't know what exactly they are doing but this operation is taking a huge amount of time, every time i build c::b (and this is are a lot of times lately)

for the moment i deleted the command from the pre-build steps, but this gives me a diff for the project file and i wanted to avoid this.

So my questions are:
* What does this executable
* Is there a way to cache the results, so that it hasn't to rescan every time i build c::b?

greetings

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Building git repo
« Reply #1 on: December 01, 2013, 12:25:00 am »
Is this on windows? On my linux it doesn't take much time.
The source for this executable is in build_tools/autorevision/autorevision.cpp probably you can modify it to fix this problem and then provide a patch.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: Building git repo
« Reply #2 on: December 01, 2013, 12:32:11 am »
yes windows -.-
here it takes at least 15min. The slowness seems to be in git svn info. I will look into it...

Offline stahta01

  • Lives here!
  • ****
  • Posts: 7591
    • My Best Post
Re: Building git repo
« Reply #3 on: December 01, 2013, 01:28:11 am »
yes windows -.-
here it takes at least 15min. The slowness seems to be in git svn info. I will look into it...

What version/build of git for Windows are you using?

I am trying this myself on Windows and forgot where I found the git svn I found that worked.
(Had hard drive semi crash on me.)

Tim S.
C Programmer working to learn more about C++ and Git.
On Windows 7 64 bit and Windows 10 64 bit.
--
When in doubt, read the CB WiKi FAQ. http://wiki.codeblocks.org

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: Building git repo
« Reply #4 on: December 01, 2013, 11:02:28 am »
1.8.1 from msys

Offline thomas

  • Administrator
  • Lives here!
  • *****
  • Posts: 3979
Re: Building git repo
« Reply #5 on: December 01, 2013, 01:30:11 pm »
The issue is that you are trying to build a non-supported source tree, if you ask me.

The autorevision tool does nothing but call subversion to find out the current revision of the checked out sources. It then places the value as an integer literal and as a character constant into a file. If you can figure out correctly what svn revision your checked out copy refers to, you can edit that file by hand and put in the respective number.

The reason for the existence of this tool is that once upon a time there was the desire to have the revision number prominently displayed, since whenever someone reported a problem and was asked what revision they used, the answer was uniformly: "Uh, what? I have no idea... where can I see this?"

Now the thing is, git doesn't have revisions (or does it, in the mean time?), so it's kind of hard for subversion to find it the current revision. I don't see how you could patch the tool in a meaningful way either because not only is a git commit .... what is it called, checksum (?) like  773dea65156909838fa6c22825cafe090ff8030 not valid, and unwieldy, and it also probably won't play well with the layout on the start page.
"We should forget about small efficiencies, say about 97% of the time: Premature quotation is the root of public humiliation."

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5915
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Building git repo
« Reply #6 on: December 01, 2013, 02:53:15 pm »
I use git-svn (portablemsysgit+tortoisegit under Windows)for half a year, I don't have such issue. But obviously I don't put the bin folder of msysgit in PATH, so my build C::B always show a "0" in the revision dialog. :)

Basically, the msysgit command can only be used in a bash shell (in msys prompt), not the normal Windows command line.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Building git repo
« Reply #7 on: December 01, 2013, 02:54:59 pm »
This is how we do it for the autotools revision number:
Code
elif git log --max-count=1 >/dev/null 2>&1; then
        echo "Using 'git log --graph' to get the revision"
        REV=`git log --graph | grep 'git-svn-id' | head -n 1 | grep -o -e "@\([0-9]*\)" | tr -d '@ '`
        LCD=`git log --date=iso --max-count=1 | grep -o -e "Date: \(.*\)" | cut -d ' ' -f 2- | sed 's/^ *//' | cut -f -2 -d ' '`
else

What this does is to find the last svn commit and extract the svn rev from its commit message.
Probably, it can be done the same way in the autoversion tool, too.
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: Building git repo
« Reply #8 on: December 09, 2013, 11:48:58 pm »
Hi.
i have found a possible speed up for this problem. If i use  "git svn find-rev" instead of "git svn log" then i have a speed up of 6000% ;)

patch is attached

there is only a different date format if you use the git version, because i can't modify the git date output to look like the svn output...

greetings

ps. this is again a git patch...
pss. Am i really the only one with the speed problems of "git svn log" ? git version 1.8.4.msysgit.0  (it doesn't work from msys as from the cmd)

[edit] The patch had a view bugs...
« Last Edit: December 10, 2013, 12:51:42 am by BlueHazzard »

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5915
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Building git repo
« Reply #9 on: January 01, 2014, 01:27:51 pm »
Hi.
i have found a possible speed up for this problem. If i use  "git svn find-rev" instead of "git svn log" then i have a speed up of 6000% ;)

patch is attached

there is only a different date format if you use the git version, because i can't modify the git date output to look like the svn output...

greetings

ps. this is again a git patch...
pss. Am i really the only one with the speed problems of "git svn log" ? git version 1.8.4.msysgit.0  (it doesn't work from msys as from the cmd)

[edit] The patch had a view bugs...

Follow the instructions of README.portable file under PortableGit-1.8.5.2-preview20131230
I simply add the folder: E:\code\msys\PortableGit-1.8.5.2-preview20131230\cmd to the PATH. (there is a file named git.exe in this folder)

I'm using a git-svn mirror(I use git-svn to checkout C::B svn about half year ago, not the full svn revision), I found the trunk code of autorevision.cpp works fine, it can detect the correct svn revision number my local git branch based, but with your patch, I get the svn revision number 0.

I don't see much time difference before and after your patch.




If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Building git repo
« Reply #10 on: January 18, 2014, 04:05:39 pm »
I am seeing this on my Ubuntu 13.10 machine. Really odd!

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: Building git repo
« Reply #11 on: January 18, 2014, 04:27:19 pm »
does the patch work?

Offline Jenna

  • Administrator
  • Lives here!
  • *****
  • Posts: 7255
Re: Building git repo
« Reply #12 on: January 18, 2014, 09:09:29 pm »
does the patch work?
The patch can not work, if the last commit is just a local one, because in this case the hash is not related to an svn revision.

Offline BlueHazzard

  • Developer
  • Lives here!
  • *****
  • Posts: 3353
Re: Building git repo
« Reply #13 on: January 18, 2014, 09:24:56 pm »
yes. then in prints out 0
but if you have a local commit, then you don't have a valid svn number... so...

[edit]
but the real question is: why is it on some machines slow (what is somehow expected, because it has to scan the whole tree) and on other so fast...

greetings
« Last Edit: January 18, 2014, 09:26:58 pm by BlueHazzard »

Offline dmoore

  • Developer
  • Lives here!
  • *****
  • Posts: 1576
Re: Building git repo
« Reply #14 on: January 19, 2014, 04:19:44 am »
Annoyingly the delay isn't even consistent on my Ubuntu machine. Sometimes it's slow, sometimes not.

Offline EnterTheNameHere

  • Multiple posting newcomer
  • *
  • Posts: 19
Re: Building git repo
« Reply #15 on: January 23, 2014, 12:33:20 pm »
Hi,
every time i build c::b from a git repo (https://github.com/obfuscated/codeblocks_sf/tree/no_ui_sdk) i get
Code
svn: E155007: '[...]\src' is not a working copy
and autorevision starts a huge tree of git/perl executables. I don't know what exactly they are doing but this operation is taking a huge amount of time, every time i build c::b (and this is are a lot of times lately)

I use git-svn (portablemsysgit+tortoisegit under Windows)for half a year, I don't have such issue. But obviously I don't put the bin folder of msysgit in PATH, so my build C::B always show a "0" in the revision dialog. :)

Basically, the msysgit command can only be used in a bash shell (in msys prompt), not the normal Windows command line.

I'm building Alpha's cc_interface git branch and this issue appears there too. I found out from autorevision.cpp that the command "git svn info" (line 160) causes this long operation, or maybe even infinite operation, I terminated autorevision.exe after 20 minutes. I tested it in msys prompt too, but it's the same behavior.

As a quick fix I just removed the pre-build command from sdk build and provided custom autorevision.h with "const unsigned int svn_revision = 0;" etc. Can this cause any issues for Code::Blocks and it's plugins, or is the svn revision only used to display it/create changelog?

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Building git repo
« Reply #16 on: January 23, 2014, 09:22:27 pm »
As a quick fix I just removed the pre-build command from sdk build and provided custom autorevision.h with "const unsigned int svn_revision = 0;" etc. Can this cause any issues for Code::Blocks and it's plugins, or is the svn revision only used to display it/create changelog?
No, but it should stay away from official svn trunk:)
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 5915
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: Building git repo
« Reply #17 on: January 24, 2014, 07:58:21 am »
@ EnterTheNameHere
It works fine here, you can read this post: http://forums.codeblocks.org/index.php/topic,18635.msg128345.html#msg128345
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline jondo

  • Single posting newcomer
  • *
  • Posts: 2
Re: Building git repo
« Reply #18 on: December 12, 2014, 02:51:33 pm »
[...]
I don't see how you could patch the tool in a meaningful way either because not only is a git commit .... what is it called, checksum (?) like  773dea65156909838fa6c22825cafe090ff8030 not valid, and unwieldy, and it also probably won't play well with the layout on the start page.

It is usual to abbreviate this Git commit hash to the first e.g. 6 characters: "773dea". This carries enough info to identify the revision and is short enough to replace any SVN version.
To reduce the effect that such commit hashes do not carry any time information, it makes sense to increase the version number more often (e.g. via semantic versioning) and to display a build date.

Offline oBFusCATed

  • Developer
  • Lives here!
  • *****
  • Posts: 13413
    • Travis build status
Re: Building git repo
« Reply #19 on: December 12, 2014, 08:59:23 pm »
Patches welcome. But keep in mind that the official VCS is still SVN!
(most of the time I ignore long posts)
[strangers don't send me private messages, I'll ignore them; post a topic in the forum, but first read the rules!]