I have been using Clojure, ClojureScript, lein, shadow-cljs, re-frame, reagent, Emacs, and CIDER to work on a Clojure/ClojureScript dynamic web app project.
Usually, I build the project by executing shadow-cljs watch app. It works fine. I can use the application and watch changes.
Currently, I am working on a Continuous Integration project via GitHub Actions. In this new environment, I wanna use a different command: shadow-cljs compile app.
But, I am having problems. Both on GitHub Actions env and in my local env (which is reproducing the steps on GitHub Actions).
The result returned by the command is quite weird. First, it says the compilation went well, Build completed, and the terminal displays some non-harmful warnings:
shadow-cljs compile app
shadow-cljs - config: /Users/pedro/projects/my_project/shadow-cljs.edn
WARNING: random-uuid already refers to: #'clojure.core/random-uuid in namespace: portal.runtime.browser, being replaced by: #'portal.runtime.browser/random-uuid
[:app] Compiling ...
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
WARNING: abs already refers to: #'clojure.core/abs in namespace: day8.re-frame-10x.inlined-deps.garden.v1v3v10.garden.color, being replaced by: #'day8.re-frame-10x.inlined-deps.garden.v1v3v10.garden.color/abs
WARNING: abs already refers to: #'clojure.core/abs in namespace: garden.color, being replaced by: #'garden.color/abs
[:app] Build completed. (1121 files, 27 compiled, 8 warnings, 45.85s)
------ WARNING #1 - :redef -----------------------------------------------------
(... omitted ...)
Unfortunately, the terminal stays like this forever. It behaves like the watch command... But, compile is not supposed to be like this.
The terminal does not finish the process. There is no exit. Even though the build is complete, as stated in the last line before WARNING.
Feels like a deadlock situation. The terminal is still responsive because I can cancel the process with Control-c. But, besides canceling it, it is frozen. This can be catastrophic on GitHub actions since you are going to be paying for an ongoing process.
On shadow-clj.edn, I tried commenting out compilation-options:
;:compiler-options {:optimizations :none}
But it did not make any difference. The same problem happens with shadow-cljs release app.
After finding a GitHub issue with a similar problem, I also tried:
:js-options {:resolve {"highlight.js" {:target :npm :require "highlight.js/lib/core"}}}
One of my hypotheses is that highlight.js is causing some trouble while parsing strings and/or files...
I am also afraid the problem could be some REPL being fired up by some dependency and blocking the process for the exit... But I am not sure where to look for it.
Why is this happening? What could be causing this? How can I solve it?
;;;;
UPDATE:
After #thomasheller kind answer, I tried some things. I believe all of the hints can be discarded for the present situation, except for the macros:
1 - The project does not use user.clj
2 - Build hooks is the default setting which is basically a comment:
:build-hooks [;; this will create a build report for every release build
;; which includes a detailed breakdown of the included sources
;; and how much they each contributed to the overall size:
#_(shadow.cljs.build-report/hook
{:output-to "build-reports/report.html"})]
3- There is no shadow-cljs start, server, or watch process going before trying to compile. The problem of an endless compile app happens just after re-starting my MacBook, on the CI, in others' people's PC... So, it is unrelated.
Ok. Now, let's talk about the macros... The project's main repository has 6 defmacros - according to a git grep (I have not checked the dependencies).
I am suspicious about one of the macros which is involved in reading files:
(defmacro slurp [file & [default]]
(if (.exists (io/file file))
(clojure.core/slurp file)
default))
Why does this exist? Well, it is being used inside another macro that reads config files related to environment variables:
(defmacro read-open-config [env-var]
(clojure.edn/read-string (slurp (str "config/" (System/getenv env-var) "-open.edn"))))
We have multiple config files for different purposes (I know it is not the standard practice...).
It feels like too much dynamicity...
First of all a few clarifications.
compile produces a development build, no optimizations are applied. Setting anything related to that does nothing.
CI systems should likely use release. This avoid including all the development related code (eg. highlight.js, re-frame-10x).
A dependency cannot "fire up a REPL". The shadow-cljs "server mode" provides it, which compile or release don't normally enter.
highlight.js errors would fail earlier with a visible error, you would not get to "Build completed."
This could be happening for a variety of reasons, very hard to debug without seeing the build config/setup. A few guesses:
You have a user.clj on the classpath, which starts additional "stuff" when the process is launched and this namespace is loaded? Clojure will load this unconditionally and shadow-cljs cannot prevent whatever that may do.
You have configured :build-hooks that launches additional stuff. It could launch an addition css watch process that doesn't exit?
You use a macro that launches additional stuff without shutting it down?
You previously started a shadow-cljs start, server or watch process? Coupled with the above that may then wait for the server to stop. On your local machine is the watch maybe still running? You can prevent the attempted server connect via npx shadow-cljs release build --force-spawn, but that is only useful if it is actually running.
Normally shadow-cljs will shut down after compile or release. I'm not aware of anyone ever having any issues with this after a successful build. The issue you linked is not related to yours, since that never got to a completed build.
After a long research process, it was discovered that the problem came from tests inside a private dependency. Apparently, some tests with asynchronous processes profile were holding the compilation process from finishing.
After commenting-out some olds tests, the compilation happened as expected.
Related
I'm trying to set up a Windows Server-based continuous integration server to completely build and package an Unreal Engine 4 project. The vast majority of the process works, but at the content cooking stage I keep running into the following errors:
********** COOK COMMAND STARTED **********
Running UE4Editor Cook for project C:\workspace\CEIT_ingame-native-plugins_PR-44\sampleProjects\unreal\ShooterGame26\ShooterGame.uproject
Commandlet log file is C:\Unreal426\Windows\Engine\Programs\AutomationTool\Saved\Cook-2021.07.05-13.56.23.txt
Running: C:\Unreal426\Windows\Engine\Binaries\Win64\UE4Editor-Cmd.exe C:\workspace\CEIT_ingame-native-plugins_PR-44\sampleProjects\unreal\ShooterGame26\ShooterGame.uproject -run=Cook -TargetPlatform=WindowsClient -fileopenlog -ddc=DerivedDataBackendGraph -unversioned -abslog=C:\Unreal426\Windows\Engine\Programs\AutomationTool\Saved\Cook-2021.07.05-13.56.23.txt -stdout -CrashForUAT -unattended -NoLogTimes -UTF8Output
LogInit: Display: Running engine for game: ShooterGame
LogModuleManager: Warning: ModuleManager: Unable to load module 'C:/Unreal426/Windows/Engine/Binaries/Win64/UE4Editor-OpenGLDrv.dll' because the file couldn't be loaded by the OS.
LogModuleManager: Warning: ModuleManager: Unable to load module 'C:/Unreal426/Windows/Engine/Plugins/Lumin/MagicLeap/Binaries/Win64/UE4Editor-MagicLeap.dll' because the file couldn't be loaded by the OS.
Took 14.257796s to run UE4Editor-Cmd.exe, ExitCode=1
ERROR: Cook failed.
(see C:\Users\jenkins\AppData\Roaming\Unreal Engine\AutomationTool\Logs\C+Unreal426+Windows\Log.txt for full exception trace)
AutomationTool exiting with ExitCode=25 (Error_UnknownCookFailure)
BUILD FAILED
Specifically, UE4Editor-OpenGLDrv.dll and UE4Editor-MagicLeap.dll cannot be loaded, but there's not any clear indication as to why this is, just that "the file couldn't be loaded by the OS". The log files written to disk don't tell me much more than the information above. I've verified that both DLLs are actually present on the CI server, so I suspect that there is some other sub-dependency missing.
I've tried running Dependencies on the Unreal executable and the DLLs mentioned in the logs to work out which DLLs might be missing on the server machine itself, but this takes over three hours to run to completion, so is a bit awkward and time-consuming to do repeatedly. I've followed the advice regarding missing dependencies from this page, and have gone through all of the likely DLLs that were reported as not found by the Dependencies utility (mostly DirectX/OpenGL related ones), but the build still fails and I'm running out of ideas.
Is there any easy way in Windows to work out exactly why a DLL fails to load? I seem to remember that Windows DLL loading error messages are nowhere near as informative as on Linux, but perhaps there's a tool or an easier method to work it out that I'm not familiar with.
EDIT: I've narrowed things down somewhat - if I attempt to load glu32.dll completely dynamically in a program of my own, I get the load error Could not load C:\Windows\System32\glu32.dll: The specified procedure could not be found. As this is on the load attempt, rather than attempt at looking up a function, it implies that some procedure is missing on a sub-dependency of glu32.dll, but I don't know how I'd go about identifying which one it is.
You can try to delete Engine/Intermediate and click GenerateProjectFiles.bat to regenerate the whole project if you use UE Source code to start up, see UE documentation and rebuild with Visual Studio.
If you use the UE4-Editor to start up lacking dll, just add dependency within YourProject.build.cs like a third party, see UE document.
I'm just trying to get my foot inside the OSX world after recently getting a MAC.
Over the past months i haven't successfully built a working VST 2.4 yet. I simply dont get why: the projects in the vst examples work (somewhat) out of the box, but my own projects fail to work.
I've mirrored every build setting exactly (including info.plist and pkginfo), double checked that the contents of the vst.app is identical, correctly gets build as vst with correct extensions etc., and the code is virtually the same, however my vst doesn't get recognized in any of the hosts i tried.
The commandline even is identical for the build.
I've tested my VST with the included minihost vst tester, and it passes and works - but still wont be recognized. I even checked the exported symbols with nm and they look correct (ie. createEffectInstance is correctly exported).
What gives? There must be some hidden build setting somewhere that i haven't discovered that seems to disqualify my VST.
Probably the most obvious but overlooked setting: are you building as 32 or 64 bit? You need to make sure that the bitness matches your host, otherwise the plugin won't be loaded (which, btw, might explain why the plugin could load in your self-built minihost and not another sequencer). To ensure compatibility with most hosts, I'd go for 32-bit build.
Also, here's a tutorial I wrote on the subject awhile back. However, you claim that you are doing everything correct with the Info.plist and whatnot, but perhaps you missed a small step:
http://teragonaudio.com/article/Making-a-VST-plugin-from-scratch-with-Xcode.html
Another potentially useful tool is MrsWatson (disclaimer: I'm the author of that program). It's a command-line VST host which can be used to provide diagnostic information about VST's, and also is designed for plugin testing and debugging. Because of the 32/64 bit difficulties with plugins, on Mac OSX the program ships separate 32 & 64 bit binaries rather than using a universal binary.
You should try running the following command on your plugin:
mrswatson --verbose --plugin /path/to/wherever/you/put/the/plugin.vst --display-info
If you see a list of parameters and other info, then it should be kosher and able to be loaded in most sequencers. Hope this gets you on the right track!
Try using your debugger to watch what Dispatcher() calls your host makes. Check if there are any differences between your plugin and the included example projects. In my experience a host will usually abort loading a plugin immediately after a Dispatcher() call raises an exception or returns a result the host doesn't like for whatever reason.
I am trying to debug an intermittent parallel build issue in my cmake build system around some generated files. It is however difficult to reliably test or reproduce the issue.
Does anyone know any way to exacerbate or sensitise such issues? Or other strategies for debugging them?
It is likely a missing add_dependencies to force one target to build completely before another begins, or an add_custom_command output that is used in more than one library.
If both libraries start building at the same time, and they both trigger running the custom command at the same time, then you'll get two competing custom commands running, and they may overwrite each other's results, or intermingle results.
Is your code public? Can you post it for others to inspect?
One good strategy is simply exposing it to other developers for "more eyes"...
I have a custom framework that, following the advice in Apple's Framework Programming Guide >> Installing your framework I install in /Library/Frameworks. I do this by adding a Run Script build phase with the following script:
cp -R build/Debug/MyFramework.framework /Library/Frameworks
In my projects I then link against /Library/Frameworks/MyFramework and import it in my classes like so:
#import <MyFramework/MyFramework.h>
This works very well, except that I always see the following message in my debugger console:
Loading program into debugger…
sharedlibrary apply-load-rules all
warning: Unable to read symbols for "/Users/elisevanlooij/Library/Frameworks/MyFramework.framework/Versions/A/MyFramework" (file not found).
warning: Unable to read symbols from "MyFramework" (not yet mapped into memory).
Program loaded.
Apparently, the compiler first looks in /Users/elisevanlooij/Library/Frameworks, can't find MyFramework, then looks in /Library/Frameworks, does find MyFramework and continues on its merry way. So far this has been more of an annoyance than a real problem, but when runnning unit tests, gdb stops on the (file not found) and refuses to continue. I have solved the problem by adding an extra line to the Run Script Phase
cp -R build/Debug/MyFramework.framework ~/Library/Frameworks
but it feels like sello-taping something that shouldn't be broken in the first place. How can I fix this?
In the past months, I've learned a lot more about frameworks, so I'm rewriting this answer. Please note that I'm talking about installing a framework as part of the development workflow.
The preferred location for installing a public framework (i.e. a framework that will be used by more than one of your apps or bundles) is /Library/Frameworks[link text] because "frameworks in this location are discovered automatically by the compiler at compile time and the dynamic linker at runtime."[Framework Programming Guide]. The most elegant way to do this is in the Deployment section of the Build settings.
As you work on your framework, there are times when you do want to update the framework when you do a build, and times when you don't. For that reason, I change the Deployment settings only in the Release Configuration. So:
Double-click on the framework target to bring up the Target info window and switch to the Build tab.
Select Release in the Configuration selectbox.
Scroll down to the Deployment section and enter the following values:
Deployment Location = YES (click the checkbox)
Installation Build Products Location = /
Installation Directory = /Library/Frameworks
The Installation Build Products Location serves as the root of the installation. Its default value is some /tmp directory: if you don't change it to the system root, you'll never see your installed framework since it's hiding in the /tmp.
Now you can work on your framework as you like in the Debug configuration without upsetting your other projects and when you are ready to publish all you need to do is switch to Release and do a Build.
Xcode 4 Warning
Since switching to Xcode 4, I've experienced a number of problems with my custom framework. Mostly, they are linking warnings in GDB that do not really interfere with the usefulness of the framework, except when running the built-in unit-test. I have submitted a technical support ticket to Apple a week ago, and they are still looking into it. When I get a working solution I will update this answer since the question has proven quite popular (1 kViews and counting).
There's not much reason to put a framework into Library/Frameworks, and it's a lot of work: You'd need to either do it for the user in an Installer package, which is a tremendous hassle to create and maintain, or have installation code in your app (which could only install to ~/L/F, unless you expend the time and effort necessary to make your app capable of installing to /L/F with root powers).
Much more common is what Apple calls a “private framework”. You'll bundle this into your application bundle.
Even frameworks intended for general use by any applications (e.g., Sparkle, Growl) are, in practice, built to be used as private frameworks, simply because the “right” way of installing a single copy of the framework to Library/Frameworks is such a hassle.
The conventional way to do this is to have your framework project and its clients share a common build directory. Xcode will search for framework headers and link against framework binaries in the build folder first, before any other location. So an app project that compiles and links against the header will pick up the most-recently-built one, rather than whatever's installed.
You can then remove the cp -r and instead use the Install Location build setting to place your build product in the final location, using xcodebuild install DSTROOT=/ at the command line. But you'll only need to do this when you're finished, not every time you rebuild the framework.
Naturally, when you distribute your framework it should be installed in /Library/Frameworks; however it seems odd to me that you're doing that with the test/debug versions of your framework.
My first instinct would be to install test versions under ~/Library, as it just makes setting up your test and debug environment that much simpler. If possible, I would expect the debug/test framework to be located in the build tree of the version I'm testing, in which case it's installed as a Private Framework for testing purposes. That would make your life much simpler when it comes time to deal with multiple versions of your framework.
Ultimately, it doesn't matter where the framework is located as long as your application or test suite loads the correct version. Choose the location that makes testing/debugging/development easiest.
I read the question on 'The best way to unit test Objective-C and followed the instructions, but no matter what I do, the Unit tests do not run. Actually the entire program does not run, I get the following message.
dyld: Library not loaded: #rpath/SenTestingKit.framework/Versions/A/SenTestingKit Referenced from /Users/garethlewis/work/objc/UnitTesting/build/Debug/UnitTesting
Reason: image not found
I have set the DYLD_FALLBACK_FRAMEWORK_PATH variable, and also the XCInjectBundle as well as the DYLD_INSERT_LIBRARIES and added the variable -SenTest All.
I can't have the only installation of Xcode 3.1 that Unit testing fails on.
Can someone who has managed to get Unit Testing on Xcode 3.1 working give some details on what needs to be done. It would help so much, with what I am trying to do.
You don't need to do this stuff to just run your tests.
If you're writing tests for an application, you should just need to set the Test Host and Bundle Loader build settings for your unit test bundle target and they will be run as part of your build. If you're writing tests for a framework you don't even need to do that, just make sure your test bundle links against your framework.
I assume you're actually talking about debugging your tests, not just running them. If so, it's important to give us the following information:
what kind of tests — application or framework — you're trying to debug
what environment variables you set, and what values you set them to
what arguments you set (-SenTest All should be an argument, not environment variable)
what the full error shown in your debug console is, not just the specific failure
That will help diagnose what's going on.
At first glance, it looks like you might have a typo in your DYLD_FALLBACK_FRAMEWORK_PATH because that determines where dyld will look for the SenTestingKit.framework binary if #rpath cannot be resolved. Knowing what it's set to will probably help.
(PS - It's Xcode.)