I'm using joliver/EventStore library and trying to find a way of how to get a stream not reading any events from it.
The reason is that I want just to write some events into that store for specific stream without loading all 10k messages from it.
The way you're expected to use the store is that you always do a GetById first. Even if you new up an Aggregate and Save it, you'll see in the CommonDomain EventStoreRepository that it will first correlate it with the existing data.
The key reason why a read is needed first is that the infrastructure needs to work out how many events have gone before to compute the new commit sequence number.
Regarding your citing of your example threshold that makes you want to optimize this away... If you're really going to have that level of events, you'll already be into snapshotting territory as you'll need to have an appropriately efficient way of doing things other than blind write too.
Even if you're not intending to lean on snapshotting, half the benefit of using EventStore is that the facility is buitl in for when you need it.
I want to build a Hadoop-Job that basically takes the wikipedia pagecount-statistic as input and creates a list like
en-Articlename: en:count de:count fr:count
For that I need the different articlenames related to each language - i.e. Bruges(en, fr), Brügge(de), which the MediaWikiApi query articlewise(http://en.wikipedia.org/w/api.php?action=query&titles=Bruges&prop=langlinks&lllimit=500).
My question is to find the right approach to solve this problem.
My sketched approach would be:
Process the pagecount file line by line (line-example 'de Brugge 2 48824')
Query the MediaApi and write sth. like'en-Articlename: process-language-key:count'
Aggreate all en-Articlename-values to one line (maybe in a second job?)
Now it seems rather unhandy to query the MediaAPI for every line but currently can not get my head around a better solution.
Do you think the current approach for is feasible or can you think of a different one?
On a sidenote: The created job-chain shall be used to do some time-measuring on my (small) Hadoop-Cluster, so altering the task is still okay
Edit:
Here is a quite similar discussion which I just found now..
I think it isn't a good idea to query MediaApi during your batch processing due to:
network latency (your processing will be considerably slowed down)
single point of failure (if the api or your internet connection goes down your calculation will be aborted)
external dependency (its hard to repeat the calculation and got the same result)
legal issues and a ban possibility
The possible solution to your problem is to download the whole wikipedia dump. Each article contains links to that article in other languages in a predefined format, so you can easily write a map/reduce job that collects that information and builds a correspondence between English article name and the rest.
Then you can use the correspondence in a map/reduce job processing pagecount-statistic. If you do that you'll become independent to mediawiki's api, speed up your data processing and improve debugging.
I was reading code from one of the projects from github. I came across something called a Vectored Referencing buffer implementation. Can have someone come across this ? What are the practical applications of this. I did a quick google search and wasn't able to find any simple sample implementation for this.
Some insight would be helpful.
http://www.ibm.com/developerworks/library/j-zerocopy/
http://www.linuxjournal.com/article/6345
http://www.seccuris.com/documents/whitepapers/20070517-devsummit-zerocopybpf.pdf
https://github.com/joyent/node/pull/304
I think some more insight on your specific project/usage/etc would allow for a more specific answer.
However, the term is generally used to either change or start an interface/function/routine with the goal that it does not allocate another instance of its input in order to perform its operations.
EDIT: Ok, after reading the new title, I think you are simply talking about pushing buffers into a vector of buffers. This keeps your code clean, you can pass any buffer you need with minimal overhead to any function call, and allows for a better cleanup time if your code isn't managed.
EDIT 2: Do you mean this http://cpansearch.perl.org/src/TYPESTER/Data-MessagePack-Stream-0.07/msgpack-0.5.7/src/msgpack/vrefbuffer.h
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 3 years ago.
Improve this question
I have complex project without comments. The project is programed in Java but have more than one main class, use several .txt files like a template and use several .bat files. I don't know where to start and how to start discovering the project, because I need to make some changes in that project.
As with others I say this is a slow process.
However having done this in the past many times, this is my methodology:
Identify as many requirements that the code fulfils. This may give you the some reasons as to why certain things are the way they are when you look deeper. A common way of finding these is look for any tests that be available. The automated ones are best, but usually they're as missing as the comments.
Find the entry points to the code. These will give you places where you can poke the code to see how different inputs affect the flow. Common entry points are Code Loading 'Main' type functions, service interfaces, web page post backs etc..
Diagram the code. Look for tools that can build black/white box pictures of the code. For me this invaluable. I have on occasion printed out large listings and attacked then with markers and rulers. You're aim to create your own flow chart (mental or other wise) of the code flow.
Using the above (iteratively) build a set of outputs to the code that you think should occur, and add to these the outputs you may already know about such as logs, data files, database writes etc..
Finally if you have time, create some manual tests though preferably in automated test harnesses to verify the above. This where I start to involve the debugger to see detail in the code.
This methodology usually gives me confidence to make changes.
Note this is iterative process and can be done with portions of the code or overall as you see fit. I usually favour a top down approach to start with and then as I gain some insight I drill down till details become overwhelming and then I repeat. However this is just because my mind works in this way - you may be different. Good luck.
Find the main Main class. The starting point.
Start drawing a picture of the classes and the objects they own and the external entities they reference.
Follow all the branches until you can find a logical ending.
I've used UML reverse engineering tools in the past and while a visual picture is good, stepping through the code has always been the hardest and yet best methodology for me.
And, as you step through each piece you can add in your own comments..
I usually start with doxygen, turning on every extracting option (especially EXTRACT_ALL and EXTRACT_PRIVATE), and enable the SOURCE_BROWSER, HAVE_DOT, CALL_GRAPH and CALLER_GRAPH options (you also need to have dot installed). This gives good view of the software. For every function the calls are displayed and linked in a graph, also the sources are linked from there.
While doxygen is intended for C and C++, it also works with Java sources (set the OPTIMIZE_OUTPUT_JAVA option).
Auch. I'm afraid there is no speedy way to do this. Comment out a line (or two) -> test -> see what breaks. You could also put break statements here and there and run the debugger. That should give you some indication how you got there (ie. what the hierarchy between the classes is).
Hopefully the original developers used some patterns that you can recognize and make notes. Make lots of notes of everything. Start by trying to understand the high level structure and work down from there.
Be prepared to spend endless hours not understanding what the thing is doing.
Speak to the client and try to understand what the project is for, and what are all the things that it does. Someone somewhere had to put in some requirements for the stuff that's in there, if only in an email.
I would try to find the first entry point in the code closest to where you suspect you'll need to start making your changes, set a breakpoint, and start debugging. Check out the contents of local variables and work your way deeper as you get to become familiar with whats going on. Then, when you have a basic understanding of the area of code you're going to be working with, start fiddling with some small changes. Test your understanding of it. Try diagramming what you see happening. If you can do that confidently, you'll be able to decide if you need to go back and continue learning more about the code, or if you know enough to get done what you need to get done.
A start is to use an automated uml modeling tool (if you use eclipse you can use a plug-gin), and start creating UML diagrams of the various classes to see how they are related in a high level and visualize the code. This has helped many times
If there are log files being generated, have a look at it to understand the flow from the starting point (main class). Otherwise, put debug statements to understand the flow.
Ya, that sounds like a pretty bad spot to be in.
I would say that the best way is to just walk through the program line for line. Try to grasp the big picture in the code, and write alot of notes, both on paper and in comments in the code.
I would say, a good approach would be to generate documentation using javadoc or doxygen's class diagram feature, then as you run the code traverse through the class diagrams generated using doxygen and see who calls what. This works wonderfully for me everytime i am working on such a project.
I completely agree to most of the answers posted.
I can add to use a development tool that reverse engineering the code and create a class diagram, to have an overall picture of what is involved.
Then you need patience. But you will be a stronger and smarter developer when you'll get through...
Good luck!
One of the best and first things to do is to try to build and run the code. It might sound a bit simplistic but the problem when you take over undocumented code is that you can't even build it and run it. When have no clue were to start.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 years ago.
The community reviewed whether to reopen this question 4 months ago and left it closed:
Original close reason(s) were not resolved
Improve this question
I've always been interested in the data structures involved in an RPG (Role-Playing Game). In particular, I'm curious about dialogue and events based actions.
For example: If I approach an NPC at point x in the game, with items y and quests z, how would I work out what the NPC needs to say? Branching dialogue and responding to player input seems as trivial as having a defined script, and user input causes the script reader to jump to a particular line in the script, which has a corresponding set of response lines (much like a choose your own adventure)
However, tying in logic to work out if the player has certain items, and completed certain quests seems to really ruin this script based model.
I'm looking for ideas (not necessarily programming language examples) of how to approach all of this dialogue and logic, and separate it out so that it's very easy to add new branching content, without delving into too much code.
This is really an open question. I don't believe there's a single solution, but it'd be good to get the ball rolling with some ideas. As more of a designer than a programmer, I'm always interested in ways to separate content and code.
For example: If I approach an NPC at
point x in the game, with items y and
quests z, how would I work out what
the NPC needs to say? Branching
dialogue and responding to player
input seems as trivial as having a
defined script, and user input causes
the script reader to jump to a
particular line in the script, which
has a corresponding set of response
lines (much like a choose your own
adventure)
However, tying in logic to work out if
the player has certain items, and
completed certain quests seems to
really ruin this script based model.
Not at all. You simply factor the conditionals into the data.
Let's say you have your list of dialogues, numbered 1 to 400 or whatever like the Choose Your Own Adventure book examples. I assume each dialogue may consist of the text spoken by the NPC, followed by a list of responses available to the player.
So the next step is to add the conditionals in there, by simply attaching conditions to each response. The easiest way is to do this with a scripting language, so you have a short and simple piece of code that returns True if this response is available to the player and False if it is not.
eg. (XML format, but could be anything)
<dialogue id='1'>
<text>
Couldst thou venture forth and kill me 10 rats, perchance?
</text>
<response condition="True" nextDialogue='2'>
Verily! Naught could be better than slaying thy verminous foes. Ten ratty
carcasses shall I bring unto thee.
</text>
<response condition="rats_left_in_world() < 10" nextDialogue='3'>
Nay, brother! Had thou but ten rats remaining, my sword would be thine,
but tis not to be.
</response>
</dialogue>
In your scripting language, you'd need a 'rats_left_in_world' function that you can call to retrieve the value in question.
What if you have no scripting language? Well, you could have the programmer code an individual condition for each situation in your dialogue - a bit tedious, not all that difficult if your dialogue is written up-front. Then just refer to a condition by name in the conversation script.
A more advanced scheme, still not requiring a scripting language, might use a tag for each condition, like so:
<response>
<condition type='min_level' value='50'/>
Sadly squire, my time is too valuable for the likes of thee. Get thyself a
farm hand or stable boy to do thy bidding!
</response>
You can add as many conditions in there as you need, as long as they can be easily specified with one or two values. If all conditions are met, the response is available.
It's interesting, there's seems to be a core idea being missed here. We're having a discussion that relates to a programmer performing the task. Indeed, the code examples above are coupled to code, not content.
In game development, it's the content developers that we programmers want to empower. They will not (this is very important) look at code. Period. Now and again you get a technical artist or technical designer, and they're wonderful and don't mind it; but, the majority of content authors are not technically inclined.
I understand the question is for your own edification; but, it should be pointed out that, in industry, when we solve these types of problems our end users (the people utilizing the technology we're developing) are not engineers.
A system like this (branching dialogue) requires a representation in a tool that is relatively intuitive to use. For example, Unreal's Kismet visual scripting system could be utilized.
Essentially, the data structures (more than likely a branching tree as it's easy to represent/debug/etc.) would be crafted by a programmer, as would the nodes that represent the object in script. The system with its ability to link to world objects (more than likely also represented by nodes in visual scripting), etc. would then be crafted and the whole kitten caboodle linked together in some glorious bit of elegant code.
After all of that, a designer would actually be able to build a visual representation of the dialogue branching in the visual scripting language. This would be map-encounter specific, more than likely. Of course, you could procedurally generate these; but, that's more of a programmer desire than a designer's.
Just thought I'd add that bit of knowledge and insight.
EDIT: Noticed there's an XML example. I'm not sure what other designers/artists/etc. feel about it; but, the ones I've worked with cringe at the idea of touching a text file.
I'd venture to say that most modern games (be they RPGs, action games, anything above basic card/board games) generally consist of several components: The display engine, the core data structures, and typically a secondary scripting engine. One example which was popular for a time (and may still be; I haven't even spoken to a game developer in years) was Lua.
The decision-making you're talking about (events, conversation branches, etc) is typically handled by the secondary scripting engine, as the scripting languages are more flexible and typically easier to use for the game's designers. Again, most of the real story-driven or game-driving logic will actually happen here, where it can be swapped out and changed relatively easily. (At least, compared to running a full build of all the code!)
The primary game engine combines the data structures related to the world (geometry, etc), the data structures related to the player(s) and other actor(s) needed, and the scripts to drive the encounters, and uses all of that to display the final, integrated environment.
You can certainly use a scripting language to handle dialogue. Basically a script might look like this:
ShowMessage("Hello " + hero.name + ", how can I help you?")
choices = { "Open the door for me", "Tell me about yourself", "Nevermind" }
chosen = ShowChoices(choices)
if chosen == 0
if hero.inventory["gold key"] > 0
ShowMessage("You have the key! I'll open the door for you!")
isGateOpen = true
else
ShowMessage("I'm sorry, but you need the gold key")
end if
else if chosen == 1
if isGateOpen
ShowMessage("I'm the gate keeper, and the gate is open")
else
ShowMessage("I'm the gate keeper and you need gold key to pass")
end if
else
ShowMessage("Okay, tell me if you need anything")
end if
This is fine for most games. The scripting language can be simple and you can write more complicated logical branches. Your engine will have some representation of the world that is exposed to the scripting language. In this example, this means the name of the hero and the items in the inventory, but you could expose anything you like. You also define functions that could be called by scripts to do things like show a message or play some sound effect. You need to keep track of some global data that is shared between scripts, such as whether a door is open or a quest is done (perhaps as part of the map and quest classes).
In some games however, scripting could get tedious, especially if the dialogue is more dynamic and depends on many conditions (say, character mood and stats, npc knowledge, weather, items, etc.) Here it is possible to store your dialogue tree in some format that allows easily specifying preconditions and outcomes. I don't know if this is the way to do it, but I've once asked a question about storing game logic in XML files. I've found this approach to be effective for my game (in which dialogue is heavily dependent on many factors). In particular, in the future I could easily make a simple dialogue editor that doesn't require much scripting and allow you to simply define dialogue and branches with a graphical user interface.
I recently had to develop something for this, and opted for a very basic text file structure. You can see the resulting code and text format at:
https://github.com/scottbw/dialoguejs
There is a tradeoff between sophistication of scripting and ease of editing for non-programmers.
I've opted for a very simple solution for the dialogue, and handle triggering of related game events separately in a secondary scripting language.
Eventually I might add some way of adding "stage directions" to the text dialogue format that are used to trigger events in the secondary scripting engine, but again without needing to put anything that looks like code in the dialogue file itself.
That's an excellent questions. I had to solve that a few times for clients. We started with an XML structure quite similar to yours, and now we use JSON. You can see an example here: http://www.branchtrack.com/projects/on029pq6.json or https://dl.dropboxusercontent.com/u/11433463/branchtrack/on029pq6.json (prettify it for readability).
Full disclosure: the link above is generated in BranchTrack, an online editor for branching dialogues, and I am the CEO. Feel free to ask anything.
I recently tackled a problem like this while making Chat Mapper. What I do is graphically plot out the dialogues as nodes in a tree and then each node has a condition and a script associated with them. As you traverse through the tree and hit a node, you check the condition to see whether or not that node is valid, and if it is, you execute the script associated with that node. It's a fairly simple idea but seems to work well from our testing. We are using a .NET Lua interpreter for the scripts.
For my solution I developed a custom text file format consisting of seven lines of text per node. Each line can be a strided list or just a text line. Each node has a position number. The last digit of the number is a type, so there are 10 different types of nodes, such as fresh questions, confirmations, repeating actions based on prior results, etc.
Each dialog activation begins with a select query to the data store whose results can be compared against members of a strided list, to match up with the appropriate node. This is more brutal than an if/then but it makes the text config file smaller since you don't need any syntax besides the stride separator. I use a system of wildcards to allow for select query results to be able to be inserted into the speech of the NPC.
Lastly there are API hooks to allow custom scripts to interface in, in case the easy config file is not enough. I plan to make a web app gui in nodejs to allow people to visually script the config files :D