Related
The Problem:
I'm trying to have Puppet manage some of the details of several scheduled tasks without managing whether those tasks are enabled. To that end, I declare scheduled_task resources without any explicit enabled attributes, with the intention of communicating that whether the tasks are enabled is not to be altered by Puppet. Puppet, however, is persistent in re-enabling the tasks on every run, just as if I had specified enabled => true for each of them. How can I make it stop doing that?
Already Tried:
I've considered setting the attributes for each datacenter via hiera, but the reality is that this makes failovers and switches more complicated than necessary. I don't want to change my puppet code every time that needs to happen. I also can't shut-off the puppet-agent runs. I need to maintain the integrity of our rolling deployment system.
Providers?
I've read a little about providers, seems like I can handle the behavior there. However, I'm having a hard time figuring out where there is (if any) documentation that explains how to use them to override specific resource properties.
Notify/Subscribe:
I've thought about using notify/subscribe to only set the triggers to enabled on creation. I'm not thinking this is the right solution, because it's not one resource subscribing to/notifying another, it's properties being set on a single resource. If there's some magical way of doing this or something similar, I'd love to know.
Bottom-line: I just need puppet to create disabled tasks, and let me turn them on/off without changing the state in subsequent runs.
Is there a way to ignore resource attribute defaults in Puppet entirely?
Only by explicitly declaring a value for every attribute of every resource.
But that doesn't seem to be the question you really wanted to ask. You seem to be exploring ways to assign attribute values without specifying them as literals in your resource declarations, and I guess you're looking for some kind of layered approach, with the bottom layer replacing or overriding types' and providers' built-in defaults.
As you thought, you could conceivably do this at least in part via providers. You would need to write a custom provider for your resource type and specify that it be used by each resource instance. But don't. This way is complicated to implement and confusing to anyone who later has to read your manifests (maybe including future you).
Notify / subscribe, on the other hand, are simply the wrong tools for the job. They do not do what you seem to think they do, or perhaps you just haven't thought that idea through.
I think you're selling Hiera short and / or inappropriately minimizing the complexity of the task. Probably a mixture of both -- I'm inclined to guess that Hiera can do more for you than you appreciate, but also that the complications you envision will manifest to some extent, in some form, no matter what you do.
Nevertheless, there is an approach that seems to match pretty closely what I think you want: resource default declarations (I link to the latest docs, but this feature is present, in substantially the same form, in every Puppet version released in at least the last nine years). Thus, you might cause all scheduled_task resources to be disabled unless you explicitly say otherwise by putting this resource default declaration at an appropriate place:
Scheduled_task {
enabled => false
}
When choosing where to put such a statement, do note that, unlike anything else in modern Puppet, resource default statements have dynamic scope. The manual discusses that in somewhat more detail.
Revisited:
In light of the clarification of the question, I'll clarify that resource types' providers are where resource management behavior lives. Therefore, if Puppet's behavior on target machines is not what you want in the event of an altogether missing property declaration, then modifying the provider or writing your own are pretty much your only alternatives. Of course, if you have a support contract then perhaps you don't have to do that in-house.
If you don't want to hack on providers -- an altogether reasonable position -- then you're left with managing the properties to their desired values. Supposing that you employ Hiera effectively, however, you should not need to modify any manifests to control which servers have their tasks enabled.
Let us imagine we have an object D, containing some data. This is modified differently across two different locations, giving rise to data objects D1 and D2. Depending upon the contents, D1 and D2 may be in conflict with each other when being merged back as part of a synchronization process.
Systems such as version control systems simply point out that the two data objects are in conflict with each other and leave it upon the user to manually resolve the conflict.
However, let us now imagine a consumer-facing application, such as a note-taking application that synchronizes contents online. In this case, no user will want to manually resolve conflicts that may have arisen due to the user typing out two versions of the same note with different contents. Discarding the older object for the newer object isn't possible either, since there may be valuable content in the older object that the user wants.
How should I go about resolving such conflicts in a consumer-facing application?
Well, if you don't want manual conflict resolution, then you will have to automatically merge changes from both updates.
There is no way that works well for all applications. When you have a requirement like this, you have to carefully design the application so that automatic merging makes sense.
There are a few common approaches, and you can do one or all of them in various combinations:
1) Merge updates really fast. Think google docs -- updates are merged in real time as people edit. Operational Transformation (https://en.wikipedia.org/wiki/Operational_transformation) is a good way to understand exactly how to do that kind of merging, but it doesn't have to be as complicated as that doc. The reason this works well is that updates are small and you can tell if someone is messing with your stuff before you put a lot of work into it. Politeness fixes conflicts -- one of you will wait until the other is done with that stuff.
2) Locking. If you click the edit button on a note, make lock it so that nobody else can edit it until you're done, etc. This is old-school, and not nearly as slick as (1), but it can work in situations where you can't merge fast enough to do (1).
3) Design your data model and interface to make merged versions as nice as possible. If anyone can add notes, but a note can only be edited by its owner, then no problem, for example. Or maybe you can only edit my stuff if you ask permission first and I give it to you. As things get more complicated than that, this becomes increasingly difficult. It's not usually possible to do this well if you're not willing to make sacrifices in application functionality. You've got one thing on your side, though: It's rude to mess with someone else's work, so a lot of the things you can do look like you did them just to enforce good behavior, and users will thank you for them if you did it with finesse.
Note to readers: Bear with me. I promise there's a question.
I have a problem to solve and think to myself "Oh, I'll do it in Ruby."
$ bundle gem problemsolver
create problemsolver/Gemfile
create problemsolver/Rakefile
create problemsolver/.gitignore
create problemsolver/problemsolver.gemspec
create problemsolver/lib/problemsolver.rb
create problemsolver/lib/problemsolver/version.rb
Initializating git repo in /tmp/harang/problemsolver
Remove the comment on s.add_development_dependency "rspec" in problemsolver/problemsolver.gemspec and then
$ bundle exec rspec --init
The --configure option no longer needs any arguments, so true was ignored.
create spec/spec_helper.rb
create .rspec
New tests go into spec/ and must be in files that end in _spec.rb. For instance, spec/version_spec.rb
describe 'Problemsolver' do
it 'should be at version 0.0.1' do
Problemsolver::VERSION.should == '0.0.1'
end
end
To run specs--ignorning code-change runners like guard--is trivial:
$ bundle exec rspec
.
Finished in 0.00021 seconds
1 example, 0 failures
You can't see it, but the message is nicely color coded for quick "Did I screw up?" scanning? The things that are very good about this:
Setup was rapid, almost brainless (though figuring out which commands to invoke is not trivial).
Standardized layout of the source tree reduces the familiarization period with a new code-base, making collaboration more simple and reducing the lull time when picking up a project you've left for a bit.
A heavy reliance on tooling distributes best-practices through the community, roughly at the speed of new project creation.
Adding coverage tools, code watchers, linters, behavior test tools and others is no more difficult.
This stands unfavorably in contrast to the situation if one thinks, "Oh, I'll do it in Haskell."
$ mkdir problemsolver
$ cd problemsolver/
$ cabal init
Package name [default "problemsolver"]?
Package version [default "0.1"]? 0.0.1
Please choose a license:
1) GPL
2) GPL-2
3) GPL-3
4) LGPL
5) LGPL-2.1
6) LGPL-3
* 7) BSD3
8) MIT
9) PublicDomain
10) AllRightsReserved
11) OtherLicense
12) Other (specify)
Your choice [default "BSD3"]?
Author name? Brian L. Troutwine
Maintainer email [default "brian#troutwine.us"]?
Project homepage/repo URL?
Project synopsis? Solves a problem.
Project category:
1) Codec
2) Concurrency
3) Control
4) Data
5) Database
6) Development
7) Distribution
8) Game
9) Graphics
10) Language
11) Math
12) Network
13) Sound
14) System
15) Testing
16) Text
17) Web
18) Other (specify)
Your choice? ProblemSolver
ProblemSolver is not a valid choice.
Your choice? 18
Please specify? ProblemSolver
What does the package build:
1) Library
2) Executable
Your choice? 2
Generating LICENSE...
Generating Setup.hs...
Generating y.cabal...
You may want to edit the .cabal file and add a Description field.
"Great," you think, "I was so pestered I bet all the latest Haskell best-practices in software development are just waiting on my disk."
$ ls
LICENSE problemsolver.cabal Setup.hs
Allow me to summarize my feelings: :(
The generated cabal file doesn't even have a Main specified, much less instructions for setting up a rudimentary project. Still, okay. If you fart around for a bit trying to find the right search keywords you'll land on How to write a Haskell program which is okay except:
All of Haq source code gets thrown into the root directory.
The test code for Haq is only in Test.hs, is only QuickCheck and has no facility for continuing the project with split-file tests.
All of this has to be manually written or copied for each new project.
Checking Real World Haskell's Chapter 11 you'll find it doesn't even mention cabal and skirts the issue of project layout entirely. None of the resources that Don Stewart kindly answers with here are addressed in either of the aforementioned and, I'll note, Mr. Stewart doesn't explain how to use any of the tools referenced.
Note that the accepted answer in Haskell testing workflow references a project that's since moved on sufficiently so as not be a good answer but does say
As cabal test doesn't yet exist -- we have a student working on it for this year's summer of code! -- the best mechanism we have is to use cabal's user hook mechanism.
Hey, okay, the cabal documentation! The appropriate section does have examples, but they're awfully contrived but don't fail to give the impression that everyone is on their own and good luck to you.
Of course, there's always test-framework that seems to be nice but it example code doesn't offer anything beyond what's seen in the wiki and is non-scalable in the sense that as soon as my program grows in complexity I'm on the hook to develop ways of dividing up tests into manageable modules. I'm not even sure what's going on with HTF and agree with Mr. Volkov's assessment.
Mr. Jelvis' comment on the linked HTF question was of particular interest to me: the Haskell tool-chain suffers, very badly, from a tyranny of small decisions. I can't actually get down to the task at hand--solving my problem in Haskell--because I'm on the hook for getting my environment just right. Why this is bad:
It's wasted effort. Unless I'm writing a test tool, I will very, very rarely care about how my tests are slurped up, only from where.
It's difficult to learn. There seems to be no singular resource for setting up a project with testing baked in, and the various sources that do exist are sufficiently diverse as to be unhelpful.
It's difficult to reproduce. With so many moving pieces to arrange I'm bound to do it differently each time.
As a corollary, it's idiosyncratic. That means its difficult to collaborate and to pick up dormant projects.
This just plain stinks.
Maybe I'm wrong, though. Does there exist some poorly advertised tool or closely developed tools to do something similar to Bundler+Rspec in the Haskell space? If not, is there a poorly advertised canonical example of modern Haskell testing with all of Mr. Stewart's referenced goodies baked right in? The project created or demonstrated:
should by convention and tooling keep test code separate from application code in a well-defined manner (in Ruby-land, Rspec tests go in spec/, Cucumber features in features/),
should not require end-users to compile and install testing dependencies
should be easily reproducible, desirably in no more than 10 minutes and
should be standardized or have the hope of standardization.
Am I wrong in believing that there's nothing at all like this in Haskell-land?
Edit0: Please note, the Ruby language's community isn't the only applicable comparison. Paul R. is correct in identifying the strong current of configuration over convention. Other languages solve the problem of getting a scalable project structure off the ground in other ways:
C :: This language is venerable and so well-documented that you'll have trouble figuring out which well-documented approach to take. No tooling as such.
Java :: Configuration over convention: you're bound into it at the compiler level. Many tools and very well documented.
Scala :: Strong tool support.
Erlang :: Venerable and loosely documented if you know what you're looking for. Arguably configuration over convention if you're using rebar or are otherwise targeting the OTP.
Paul R.'s solution of using a custom template works great if, like C, there's sufficient documentation to compile such a thing. This still runs into issues that I attempted to identify explicitly in the post, but it's workable. Haskell's best offering--that I'm aware of--is "How to write a Haskell program" but falls short of being more than the equivalent of dumping a lone Cub Scout off in the woods with a flashlight and a flask of water.
Also, yes, Static Types are great and do solve many problems that would otherwise need explicit testing. If they were an end-all solution, or mostly sufficient, even, the snap-framework would not be so thoroughly tested. (Arguably "Copy snap-core." is an answer to my question.)
There's currently no one single way to set up a testsuite. Hopefully, people will standardize on cabal test, which is out-of-the box. In fact, both HUnit and QuickCheck are also provided with the Haskell Platform, and so setting up tests doesn't require downloading any extra dependencies.
You're correct that an old accepted answer doesn't provide information on cabal test. I edited it, and now it does! You're also probably correct that the linked page on the Haskell wiki (also written before cabal test became available) doesn't provide information on current testing best practices. It's a wiki, and I encourage folks to edit it! Note that the page does, however, provide a link to another page that describes how one might structure a more complex Haskell project.
tldr; Use cabal test. I'm fond of test-framework, which you can integrate with cabal test should you so desire. Sorry that cabal test is sort of new and not all the resources we have (generally community editable) have been updated to point to it and describe how to use it. Updating lots of resources and creating tutorials is the job of a community. We should probably do a better job promoting lots of the new awesome tools introduced to the Haskell ecosystem in the last few years.
There are many points here. First, there is a comparison of convention-over-configuration with explicit configuration. In the ruby land, the former is often prefered. In my experience, although it works great for do-a-{blog|social-thing|gem|library}-in-5-minutes-screencast and quick experiments, it has much less value in your real projects (more than 5 minutes) as init time gets quickly amortized. Also, there is a reason why tools provide configuration facilities : there are many different needs and usages. So my advice to your cabal-init problem is : make your own template file. Put stub for everything you need, with great comments, and use it whenever you need it.
Regarding tests, the landscape is quiet different between ruby and haskell. In ruby, one can write foo do { oh dear I am typing nonsense here } and there is no other way to catch this nonsense than actually running the code. So automated tests are absolutely required. In the haskell land however, there is a great static analysis of your code coupled with a very sane paradigm (purely functional non-strict), and after years of using it, I'm still surprised haw hard it is to write nonsense without being immediately caught by the compiler. I do ruby at work as well, and really, 90% of my tests are poor-man manual "static checks".
Still, there is room for wrong design or corner-case errors, that's why quickcheck exists. It will automatically (yes, really automatically) find corner-case errors and help you a lot find design errors. You can still write unit tests with one of existing packages if you need manual checks.
So my conclusion here is : don't be surprised to find shadow everywhere if you shade ruby light on the haskell land. Things are very different over here, and need to be experienced to grab power. That doesn't mean that everything is perfect, actually improving the toolchain is a commonly expressed wish. Just the points you raised are not really problematic, and really don't deserve some vocabulary you picked. Try first, judge after :)
Are there any general rules on how to realiably locate OEPs (Original Entry Points) for packed .exe files, please? What OEP clues are there to search for in debugged assembly language?
Say there is a Windows .exe file packed with PC-Guard 5.06.0400 and I wish to unpack it. Therefore, the key condition is finding the OEP within the freshly extracted block of code.
I would use the common debugger OllyDBG to do that.
In the general case - no way. It highly depends on packer. In the most common case packer may replace some code from OEP by some other code.
This depends solely on the packer and the algorithms its using pack and/or virtualize code. Seeing as you are using ollydbg, i'd suggest checking out tuts4you, woodmanns and openrce, they have many plugins (iirc there is one designed for finding oep's in obfuscated code, but i have no clue how well it performs) and olly scripts for dealing with unpacking various packers (from which you may be able to pick up hints for a certain type of packer), they also have quite a few papers/tutorials on the subject as well, which may or may not be of use.
PC Guard doesn't seem to get much attention, but the video link and info here should be of help (praise be to Google cache!)
It's hard to point out any simple strategy and claim that it will work in general, because the business of packer tools is to make OEP finding a very hard problem. Besides, with a good packer, finding the OEP is still not enough. That being said, I do have some suggestions.
I would suggest that you read this paper on the Justin unpacker, they use heuristics that were reasonably effective at the time, and that you might be able to get some mileage from. They will at least reduce the number of candidate entry points to a manageable number:
A study of the packer problem and its solutions (2008)
by Fanglu Guo , Peter Ferrie , Tzi-cker Chiueh
There are also some web-analysis pages that can tell you a lot about your packed program. For example, the malware analyzer at:
http://eureka.cyber-ta.org/
Here's another one that is currently down, but has done a reasonable job in the past, and I presume will be up again soon):
http://bitblaze.cs.berkeley.edu/renovo.html
Is there a trick to localize syntax errors? I occasionally run into this situation when editing existing code. Here's a latest example, do you see an efficient way to find the error? (I found the error already, but not efficiently)
http://yaroslavvb.com/upload/save/syntax-error.nb
May be, I am missing something, but in your example it was quite easy: once I put your code into a notebook and tried to run, the usual orange bracket appeared, and when you expand the messages, it states very clearly that the problem was in an empty parentheses in your vertexFormula function. In my experience, most of the time, the orange box gives enough hints.
Another great way which I use on a daily basis it through the code highlighting in Workbench. It immediately highlights syntax errors, plus you have a very powerful Eclipse-based navigation both for a single package and multiple packages. It might seem as you lose some flexibility by going to Workbench from an interactive FrontEnd development, but I found the opposite (or may be this is the revenge of my enterprise Java background): you still can have your notebook(s) in a Workbench project, where you do the initial development, but then they get attached to the project and a number of packages that you already developed and use. The transition from notebook to a package could not be easier, since you can continue use that notebook after you transfer the code into a package, and you don't even have to worry about loading the package, as long as you do it just once inside a project. Overall, I find the Workbench - based development much more fun as soon as your project goes over some critical mass (I'd say, perhaps around 1000 loc, but ymmv). But, complete new and independent chunks of functionality I still prefer to prototype entirely in the FrontEnd.
If you adhere to some specific coding standards in your code, or work with code which does, you might be able to develop some simple partial parser which at least breaks the code into complete chunks (function defenitions, Module-s etc, CompoundExpressions). Then, you could use ToExpression (say, Map it on a list of string code chunks returned by your partial parser), to see which piece of code is problematic (it will return $Failed on it). But if you use Workbench, this is unnecessary altogether.