I'm trying to research information about how to implement the memory management system in a DBSM. I know the theoretical part, but can't find anything about bypassing the memory management system in Linux, windows nor mac OX.
I'm trying to implement an efficient DBMS myself for fun. Which should be able to run on any common OS. In Database Systems The complete book, they discus the importance on fitting the relations/tables in adjacent sectors/blocks on hardware for faster read/write. But i can't seem to find anything when crawling the web. Is there a way to bypass the memory management system, such that I can write to specific locations on disk in such that the data saved there are placed on continuously blocks and so that the file system can recognize the data as a file?
I know that this is not a general coding question.
I'm used to write in C, C#, F#, ML, Python and a little gnu assembly.
For clarification I do not mean which DBMS should I use, I'm trying to implement the the DBSM myself.
Related
I'm now a bit experienced with using OpenGL, which I started using because it's said that it is the only way to invoke video card functions. (besides DirectX - which I like less than OpenGL)
For programming (e.g. in C/C++) the OS gives many APIs, like functions for printing. But these can also be bypassed, by coding in Assembly-language - and call much lower level APIs (which gain speed) or direct CPU calls.
So I started wondering why this wouldn't be possible on the video card. Why should an API like OpenGL or DirectX be needed? The process going on with those is:
API-call >
OS calls video card (with complex opcodes, I think) >
video card responses (in complex binary format) >
OS decodes this format and responses to user (in expected API format)
I believe this should decrease the speed of the rendering process.
So my question is:
Is there any possibility to bypass any graphical API (under Windows) and make direct calls to the video card?
Thanks,
Dennis
Using assembly or bypassing an api doesnt automatically make something faster, often slower as you dont know what the folks that wrote the library know.
it is absolutely possible yes, those libraries are just processor instructions that poke and peek at registers and ram, and you could just as easily poke and peek at registers and ram. The first problem is can you get that information, sure, you can look at the linux drivers or other open source resources. Second, much of the heavy lifting today is done in the graphics chip by logic or graphics processors, so the host is just a go between and not necessarily the bottleneck if there is a bottleneck. And yes you can program the gpus depending on your video card/chip, etc.
You need to determine where the bottleneck really is, if there really is one, maybe the bus is your problem, maybe the operating system is your problem, or the compiler, or the hard disk or the system memory, the processor and architecture itself, caches, etc. At the same time how will you ever learn how to find these things unless you try.
I recommend getting rid of windows completely, no operating system, go bare metal. Take the linux and other open source resources plus anything you can get from the vendor and get closer to the metal. You will also need a lot of info about the pci/pcie bus and bridges, dma controllers, everything in the path. If you dont want to go that low then use linux or bsd or some other command line environment where it is well known how to take over the video system, and take over the video system while retaining an operating system and a development environment (vi/emacs, gcc).
if that is all way too advanced, then I recommend, dabbling in simple gpu routines to get a feel for how the video card works at least at some level and tackle this learning exercise one step at a time.
I want to write something like DaemonTools: a software that presents itself to the system as a real device (a DVD-ROM in the previous example) but it reads the data from a file instead. My requirement is not limited to DVD-ROM. The first goal is a joystick/gamepad for Windows.
I'm a web developer, so I don't know from where I could start such a project. I believe it will have to be written in C/C++, but other than that, I have no clue where to start.
Did anyone tried something like this and can give me some starting tips ?
Most drivers are written in either C or C++, so if you don't know those languages reasonably well, you'll want to get familiar with them before you start. Windows programming uses a lot of interesting shortcuts that might be confusing to a beginner - for example PVOIDs (typedef void* PVOID) and LPVOIDs (typedef void* far LPVOD;). You'll need to be happy with pointers as concepts as well as structures because you'll be using a lot of them. I'd suggest writing a really straightforward win32 app as an exercise in getting to grips with the Windows style of doing C/C++.
Your next port of call then is to navigate the Windows Driver Kit - specifically, you'll need it to build drivers for Windows. At this stage my ability to advise really depends on what you're doing and the hardware you have available etc, or whether or not you're really using hardware. You'll need to know how to drive your hardware and from there you'll need to choose an appropriate way of writing a driver - there are several different types of driver depending on what you need to achieve and it might be you can plug into one of these.
The windows driver kit contains quite a large number of samples, including a driver that implements a virtual toaster. These should provide you with starting points.
I strongly suggest you do the testing of this in a virtual machine. If your driver successfully builds, but causes a runtime error, the result could well crash windows entirely if you're in kernel-mode. You will therefore save yourself some pain by being able to revert the virtual machine if you damage it, as well as not having to wait on your system restarting. It'll also make debugging easier as virtual serial cables can be used.
This is quite a big undertaking, so before you start, I'd research Windows development more thoroughly - check you can't do it using the Windows APIs first, then have a look at the user-mode driver framework, then finally and only if you need to, look at the kernel level stuff.
It’s the first time I submit a question in this forum.
I’m posting a general question. I don’t have to develop an application for a specific purpose.
After a lot of “googling” I still haven’t found a language/runtime/script engine/virtual machine that match these 5 requirements:
memory allocation of variables/values or objects cleaned at run time
(e.g. a la C++ that use keyword delete or free in C )
language (and consequently the program) is a script or
pseudo-compiled a la byte code that should be portable on main
operating system (windows, linux, *bsd, solaris) & platform(32/64bit)
native use of multicore (engine/runtime)
no limit on the heap usage
library for network
The programming language for building application and that run on this engine is agnostic oriented (paradigm is not important).
I hope that this post won’t stir up a Holy-War but I'd like to put focus on engine behavior during program execution.
Sorry for my bad english.
Luke
I think Erlang might fit your requirement:
most data is either allocated in local scopes and therefore immediately deleted after use or contained in a library-powered permanent storage like ETS, DETS or Mnesia. There is Garbage Collection, though, but the paradigm of the language makes the need for it not as important.
the Erlang compiler compiles the source code to the BEAM virtual machine byte code, which, unlike Java is register-based and thus much faster. The VM is available for:
Solaris (including 64 bit)
BSD
Linux
OSX
TRU64
Windows NT/2000/2003/XP/Vista/7
VxWorks
Erlang has been designed for distributed systems, concurrency and reliability from day one
Erlang's Heap grows with your demand for it, it's initially limited and expanded automatically (there are numerous tweaks you can use to configure this on a per-VM-basis)
Erlang comes from a networking background and provides tons of libraries from IP to higher-level protocols
Greetings , I have a few questions that need experts' help to demystify it.
Question:
1.)Can a GUI be created without using any framework or API like GDI and DirectX??
2.)How microsoft develop the GUI environment for his OS??
3.)Is it possible that I could get knowledge about creating a GUI which run during booting without relying on an OS??Does it require any special tools which we do not own at home but is available in microsoft??
4.)Any book recommended for me to know more about GUI development of OS like windows and linux??
Thanks for spending time reading my question , your help is much appreciated
I'm not an expert on this field but to my knowledge:
Without considering your question #3, Yes, use a programming language (PL), but if you consider PL's as framework then NO, or even if it's possible, it is certainly not practical. Even Assembly (machine language) is a programming language.
To help understand the answer to this question, I suggest you read what an operating system is and how it is created. MS Windows (depending on the version) is written using PL: C, C++, C#, and Assembly. They used what we consider now as "low-level" programming to create their GUI's.
I find this question conflicting, I believe you need an OS to run a GUI. Because an operating system is responsible for connecting your hardware together (e.g. where to output the display, where to get the inputs like keyboard and mouse, etc). If you want, create an OS yourself but again, I find this very impractical. What you can do is start learning how to create GUI from Linux or even build your own minimalistic OS from Linux from Scratch. I recommended Linux because it's free and mature (relatively stable, has tons of documentation and internet references)!
I can't recommend any specific book, but types of book: Programming Languages and Operating Systems. Also, you can find all the resources you need on the internet. You just need to know exactly what questions you are looking an answer for.
But if you simply want to create a GUI that can run on most Operating Systems, this has been the aim of Java. Java uses a virtual machine to do this. Of course there are other options too, you can research about it. If it's already available that suits your needs and it's legally free, use it. It will save you a lot of effort. :)
I have a C library, which I build as a shared object for Linux and a DLL for Windows with MinGW32. The API depends on a couple of data files (statistical models) which I'd really like to roll in with the SO/DLL so that deployment is just one file.
It looks like I can achieve this for Windows with a "resource file" compiled with windres, but then I've got to write a bunch of resource-handling code for Windows, and I'm still stuck with the files on Linux.
Is there a way to achieve the same functionality on Linux?
Even better, is there a portable solution?
It's actually quite simple on Linux and other ELF systems: http://www.linuxjournal.com/content/embedding-file-executable-aka-hello-world-version-5967
OS X has bundles, so you just build your library as a framework and put the file in the bundle.
Two potential solutions:
Phong Vo's sfio library, which is part of the AT&T Advanced Software Technology toolset, is a wonderful replacement for C stdio.h, and it will allow you to open either files or memory blocks using a single API. So you can easily convert your existing files to C initialized data to include in your DLL or SO file.
This is a good cross-platform solution, but the penalty is that the learning curve to get started is pretty high. They don't make it easy to figure out how stuff works or to take one part of their toolset and split it out for use independent of the other parts. But the good news is that if you want to adopt their U/Win system for running Unix codes on windows (all part of the same toolset), you can create DLLs and SOs using the same system.
For this kind of problem I often fall back on Lua; I can stored Lua data either in external files or within C as initialized data. This is great for distributing everything in one .so file; I do this for my students.
Again the downside is that you have to master and incorporate a new technology.
In my own work I use Lua over the AT&T stuff for these reasons:
Lua has a much smaller footprint and is designed to play well with others; with AST you really have do adopt their way of doing things.
The learning curve with Lua is much less steep; you can be productive very quickly.
Lua is dead easy to install and it's easy to get information about it. AST has its own quirky installation process shared by nobody else in the world; it's often hard to make the installation work; and it's harder to get information about it.
Using Lua has a lot of other payoffs, so the effort spent learning Lua and learning how to incorporate Lua into C codes is easy to amortize over multiple projects.