Author Topic: build bot in the github, I see one nice project  (Read 35984 times)

Offline stahta01

  • Lives here!
  • ****
  • Posts: 7670
    • My Best Post
Re: build bot in the github, I see one nice project
« Reply #30 on: August 25, 2024, 06:39:44 am »
The only way I have fixed this issue in the past was to either edit the CBP files or place a good zip command in the path.

To place a good zip command in the path; I first removed or rename the bad zip command and then via trial and error added the path to the good zip command to the PATH using the export command.

Those both worked for me over the past few years; but, neither was easy to automate.

Maybe finding the correct location in cbp2make to edit will be a better solution.

Tim S.
C Programmer working to learn more about C++ and Git.
On Windows 7 64 bit and Windows 10 64 bit.
--
When in doubt, read the CB WiKi FAQ. http://wiki.codeblocks.org

Online Wkerry

  • Multiple posting newcomer
  • *
  • Posts: 55
Re: build bot in the github, I see one nice project
« Reply #31 on: August 25, 2024, 07:36:04 am »
Have you looked at previous forum posts on github actions as per the following thread to see if it has anything useful?
https://forums.codeblocks.org/index.php?topic=24972.0

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6025
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: build bot in the github, I see one nice project
« Reply #32 on: August 25, 2024, 12:46:14 pm »
OK, I will have a try.
Still the same...
I think those commands are from the project files.

Correct, those zip command are defined in the cbp files, see the image show below.

But when those command were run from the windows command line (CMD), it works OK.

So, I'm not sure why it works badly under mingw32-make.exe, did you run mingw32-make.exe inside a msys2's mingw64 shell? or a pure CMD shell?

As Tim(stahta01) suggested, I think you have to check which zip executable the mingw32-make.exe was actually calling.

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6025
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: build bot in the github, I see one nice project
« Reply #33 on: August 27, 2024, 03:04:56 pm »
I think I found the reason why the zip command works badly.

I create a very simple makefile:

Code
# Define variables
ZIP = xyz\abc.zip
FILES = xyz\a1.txt
ZIP_CMD = zip -jq9 $(ZIP) $(FILES)

# Default target
all: zip

# Zip target
zip:
$(ZIP_CMD)

# Clean target
clean:
del $(ZIP)

# Phony targets to avoid conflicts with files
.PHONY: all zip clean

Please note that I can run the command "mingw32-make" under the mingw64 shell under msys2, and it works fine.

But if I change the "xyz\a1.txt" to "xyz\*.txt".

It will failed. my guess is that the mingw64 shell will "escape" the "\*" as a special character.

If I change to "FILES = xyz/*.txt", it works OK.

I have tried the zip command inside the mingw64 shell, or I just use the zip command (I mentioned the zip command's link before), all failed if I use the "xyz\*.txt".

Now, I see that "xyz\*.txt" works OK under Windows native command line shell.

So, the solution is: can we call the "mingw32-make.exe" inside the "Windows native CMD"?

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline stahta01

  • Lives here!
  • ****
  • Posts: 7670
    • My Best Post
Re: build bot in the github, I see one nice project
« Reply #34 on: August 27, 2024, 05:49:40 pm »
I think I found the reason why the zip command works badly.

I create a very simple makefile:

Code
# Define variables
ZIP = xyz\abc.zip
FILES = xyz\a1.txt
ZIP_CMD = zip -jq9 $(ZIP) $(FILES)

# Default target
all: zip

# Zip target
zip:
$(ZIP_CMD)

# Clean target
clean:
del $(ZIP)

# Phony targets to avoid conflicts with files
.PHONY: all zip clean

Please note that I can run the command "mingw32-make" under the mingw64 shell under msys2, and it works fine.

But if I change the "xyz\a1.txt" to "xyz\*.txt".

It will failed. my guess is that the mingw64 shell will "escape" the "\*" as a special character.

If I change to "FILES = xyz/*.txt", it works OK.

I have tried the zip command inside the mingw64 shell, or I just use the zip command (I mentioned the zip command's link before), all failed if I use the "xyz\*.txt".

Now, I see that "xyz\*.txt" works OK under Windows native command line shell.

So, the solution is: can we call the "mingw32-make.exe" inside the "Windows native CMD"?

You might try the normal make like this.

Code
MSYS2_ARG_CONV_EXCL=* make

NOTE: The "*" star might need double or single quotes around it. This option is supposed to turn off file handling magic.

See https://www.msys2.org/docs/filesystem-paths/

Tim S.
C Programmer working to learn more about C++ and Git.
On Windows 7 64 bit and Windows 10 64 bit.
--
When in doubt, read the CB WiKi FAQ. http://wiki.codeblocks.org

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6025
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: build bot in the github, I see one nice project
« Reply #35 on: August 28, 2024, 04:32:13 am »

You might try the normal make like this.

Code
MSYS2_ARG_CONV_EXCL=* make

NOTE: The "*" star might need double or single quotes around it. This option is supposed to turn off file handling magic.

See https://www.msys2.org/docs/filesystem-paths/

Tim S.

Hi, Tim, nice finding!

But why do you think "normal make" is needed. I think mingw32-make can still be used if we "disable the file path handling magic".

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline stahta01

  • Lives here!
  • ****
  • Posts: 7670
    • My Best Post
Re: build bot in the github, I see one nice project
« Reply #36 on: August 28, 2024, 01:35:01 pm »

You might try the normal make like this.

Code
MSYS2_ARG_CONV_EXCL=* make

NOTE: The "*" star might need double or single quotes around it. This option is supposed to turn off file handling magic.

See https://www.msys2.org/docs/filesystem-paths/

Tim S.

Hi, Tim, nice finding!

But why do you think "normal make" is needed. I think mingw32-make can still be used if we "disable the file path handling magic".

It depends, some things build better with one or the other of make commands and some build okay with both. Till it is tried I see no pattern [useful to predict which will happen].

Tim S.
« Last Edit: August 30, 2024, 04:06:28 pm by stahta01 »
C Programmer working to learn more about C++ and Git.
On Windows 7 64 bit and Windows 10 64 bit.
--
When in doubt, read the CB WiKi FAQ. http://wiki.codeblocks.org

Offline Grit Clef

  • Multiple posting newcomer
  • *
  • Posts: 95
  • Where there is a will, there is a way.
Re: build bot in the github, I see one nice project
« Reply #37 on: August 31, 2024, 01:23:44 pm »
It was not successful when I tried
Code
MSYS2_ARG_CONV_EXCL=* mingw32-make
. We can't use normal make, for certain reason(because those lines calling cmd /c will fail.) I always get failed using Windows' pure command line. Maybe we can only use the configure/make structure.
Edit: I'm seeing that cmd /c executed successfully when using normal make. It's hoped that the current action will be successful.
Still failed...
« Last Edit: August 31, 2024, 01:41:37 pm by Grit Clef »
-Windows 7, 32-bit
-CodeBlocks r13542, gcc 14.2.0, debug version

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6025
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: build bot in the github, I see one nice project
« Reply #38 on: August 31, 2024, 01:41:06 pm »
It was not successful when I tried
Code
MSYS2_ARG_CONV_EXCL=* mingw32-make
. We can't use normal make, for certain reason(because those lines calling cmd /c will fail.) I always get failed using Windows' pure command line. Maybe we can only use the configure/make structure.
Edit: I'm seeing that cmd /c executed successfully when using normal make. It's hoped that the current action will be successful.

Maybe, you should try something like:

Code
MSYS2_ARG_CONV_EXCL='*' mingw32-make

I haven't tried it, because I know nothing about how to debug a github action script.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline Grit Clef

  • Multiple posting newcomer
  • *
  • Posts: 95
  • Where there is a will, there is a way.
Re: build bot in the github, I see one nice project
« Reply #39 on: September 01, 2024, 05:54:08 am »
No, it didn't work.
-Windows 7, 32-bit
-CodeBlocks r13542, gcc 14.2.0, debug version

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6025
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: build bot in the github, I see one nice project
« Reply #40 on: September 01, 2024, 08:42:37 am »
No, it didn't work.

Sadly to see.

Can you use some dependency tool(such as this one: brechtsanders/pedeps: Cross-platform C library to read data from PE/PE+ files (the format of Windows .exe and .dll files)) to copy all the dll/exe files to a single folder(for example to a common bin folder) when using the configure/make steps.

I think this is the easiest way to achieve a working github working binary package release.

Thanks.
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6025
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: build bot in the github, I see one nice project
« Reply #41 on: September 14, 2024, 09:10:39 am »
Some improvements about the github build bot.

1, I think we need to build a "CbLauncher.exe" in the "bin" folder(the codeblocks.exe is in the the same bin folder), currently the configure-make method in the build bot does not generate this file. I mean by running this exe file, C::B is running in portable mode, all the configure settings were created in "bin/AppData" folder, so it won't pollute other C::B settings. For my test, I just copy a "CbLauncher.exe" from other windows build packages, and it works.

2, when I download the "Artifacts" file from the github action (for example, in my own github action here: https://github.com/asmwarrior/x86-codeblocks-builds/actions/runs/10538520726), I need to set the PATH variable to adding my local msys2/mingw64/bin path

Code
set PATH=<your_local_msys2_mingw64_bin_path>;%PATH%

by setting this, when code::blocks runs, it will find the gcc's dlls, wx's dlls, and exchnl's dlls. (suppose you install them by the pacman command)

Here is the list of dlls of the codeblocks.exe by using brechtsanders/pedeps: Cross-platform C library to read data from PE/PE+ files (the format of Windows .exe and .dll files)'s listpedeps commands:

Code
listpedeps.exe -s codeblocks.exe
[codeblocks.exe]
architecture: x86_64
machine name: AMD AMD64 (x64)
subsystem:    Windows GUI
DLL:          no
stripped:     no
file version: 0.0
minimum Windows version: 5.2
IMPORTS
libcodeblocks-0.dll
COMCTL32.dll
exchndl.dll
libgcc_s_seh-1.dll
KERNEL32.dll
msvcrt.dll
SHELL32.dll
libstdc++-6.dll
USER32.dll
wxbase32u_gcc_custom.dll
wxmsw32u_aui_gcc_custom.dll
wxmsw32u_core_gcc_custom.dll
wxmsw32u_html_gcc_custom.dll
wxmsw32u_propgrid_gcc_custom.dll
wxmsw32u_xrc_gcc_custom.dll

So, you can see that you need to copy those dlls from the msys2 mingw64's bin folder, so that you can distribute a full package:


Code
exchndl.dll


libgcc_s_seh-1.dll
libstdc++-6.dll


wxbase32u_gcc_custom.dll
wxmsw32u_aui_gcc_custom.dll
wxmsw32u_core_gcc_custom.dll
wxmsw32u_html_gcc_custom.dll
wxmsw32u_propgrid_gcc_custom.dll
wxmsw32u_xrc_gcc_custom.dl

To copy those dlls from the msys2's bin folder, you have to use another tool named "copypedeps.exe" from "brechtsanders/pedeps".

The commands are something like below:

Code
copypedeps.exe -r -n -v ./codeblocks.exe ./

The options can be explained by the "-h" help command.

Code
copypedeps.exe -h
Usage: copypedeps [-h|-?] [-r] srcfile [...] dstfolder
Parameters:
  -h -?         display command line help
  -r            recursively copy dependancies
  -n            don't overwrite existing files
  -d            dry run: don't actually copy, just display copy actions
  -q            quiet mode, only show errors
  -v            verbose mode (display copy actions)
Description:
Copies .exe and .dll files and all their dependancies to the destination folder.
Version: 0.1.14 (library version: 0.1.14)

So, can you add the extra steps like below:

1, build the CBLauncher.exe
2, use pacman to install the drmingw for the exchndl.dll and related dlls. See here: mingw-w64-x86_64-drmingw
3, use the pedeps to copy all the necessary dlls to the bin folder.

Any ideas?


EDIT:

mingw-w64-x86_64-drmingw is already installed in the current github action code.

EDIT2:

The spell checker plugin is not loaded correctly, so maybe my local msys2 does not have hunspell package installed.
« Last Edit: September 14, 2024, 10:04:18 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6025
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: build bot in the github, I see one nice project
« Reply #42 on: September 15, 2024, 05:41:20 am »
For you information:

I have create a github test project to use pedeps to copy the dlls of a hello.exe file, see here:  asmwarrior/test-github-action0

I haven't used the github action script before, so it takes one hour to tweak the script.

Finally, a artifact zip file is created with several dependency dlls.

Code
$ ls
hello.exe*  libgcc_s_seh-1.dll*  libstdc++-6.dll*  libwinpthread-1.dll*

The exe file is built from the compiler gcc, and the dlls are copied from the msys2's PATH by using the latest pedeps tool.

If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6025
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: build bot in the github, I see one nice project
« Reply #43 on: September 15, 2024, 07:00:30 am »
I have added one commit to my fork, I try to enable the pedeps tool to copy the necessary dlls.

try to use the pedeps tool to copy the dlls to the output folder

And I'm starting the github action now, let's see whether it works or not one hour later.  :)
« Last Edit: September 15, 2024, 07:19:51 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.

Offline ollydbg

  • Developer
  • Lives here!
  • *****
  • Posts: 6025
  • OpenCV and Robotics
    • Chinese OpenCV forum moderator
Re: build bot in the github, I see one nice project
« Reply #44 on: September 15, 2024, 09:01:47 am »
Some good news:

It looks the built artifact (zip file) has many dlls which are the dependencies of codeblocks.exe.

When I run the C::B, I see some warnings(see image shot below), it looks like only the dlls for the codeblocks.exe get copied, but some dlls for plugins are missing.

So, I need to loop all the plugin dll files?

Where should I copy those dependency dlls for the plugins. Because for plugin dlls, they are located in the folder:

Code
codeblocks64-zip-folder\lib\codeblocks\plugins\

While the codeblocks.exe is located in

Code
codeblocks64-zip-folder\bin\codeblocks\

One issue still remains, how to build the "CBLauncher.exe"?



EDIT:

The plugin dll's dependency dll issue is solved, see this result:
main64 asmwarrior/x86-codeblocks-builds@691cc2b
with my commit:
copy the plugin dll's dependency dll to the bin folder




« Last Edit: September 15, 2024, 11:38:18 am by ollydbg »
If some piece of memory should be reused, turn them to variables (or const variables).
If some piece of operations should be reused, turn them to functions.
If they happened together, then turn them to classes.