I recently start learning Direct Rendering Manager and tried to use the writeback connector and the libdrm.
There are some documents and codes of how the kernel implements it, but not enough of how the userspace uses the api, such as drmModeAtomicAddProperty and drmModeAtomicCommit of the writeback connector in libdrm.
I have referred to libdrm/tests/modetest; Linux API Reference; linux/v6.0-rc5/source/drivers/gpu/drm/drm_writeback.c and some patch imformation.
I used modetest to get some driver information and tried my code using the libdrm:
/* I previously get the value of writeback connector:wbc_conn_id
* and create an output framebuffer:fb_id
* and find the active crtc: crtc_id
* next to get writeback connector property*/
props = drmModeObjectGetProperties(fd, wbc_conn_id, DRM_MODE_OBJECT_CONNECTOR);
printf("the number of properties in connector %d : %d \n", wbc_conn_id, props->count_props);
writeback_fb_id_property = get_property_id(fd, props,"WRITEBACK_FB_ID");
writeback_crtc_id_property = get_property_id(fd, props,"CRTC_ID");
printf("writeback_fb_property: %d\n",writeback_fb_id_property);
printf("writeback_crtc_id_property: %d\n",writeback_crtc_id_property);
drmModeFreeObjectProperties(props);
/* atomic writeback connector update */
req = drmModeAtomicAlloc();
drmModeAtomicAddProperty(req, wbc_conn_id, writeback_crtc_id_property, crtc_id);
printf ("%d\n",ret);
drmModeAtomicAddProperty(req, wbc_conn_id, writeback_fb_id_property, buf.fb_id);
ret = drmModeAtomicCommit(fd, req, DRM_MODE_ATOMIC_ALLOW_MODESET, NULL);
if (ret) {
fprintf(stderr, "Atomic Commit failed [1]\n");
return 1;
}
drmModeAtomicFree(req);
printf("drmModeAtomicCommit Set Writeback\n");
getchar();
It turns out that the drmModeAtomicCommit failed. Was there any property set wrongly or missed? The value of two steps of addproperty is 1 and 2, and the atomic commit returned EINVAL -22 .
I've looked around but found no solution or similar question about the property set of writebackconnector.
Related
I just solved a latency issue in our infrastructure that was triggered because this code snippet here triggered a call to getaddrinfo on every run of the code:
sock = UDPSocket.open
sock.send("#{key}|#{value}", 0,
GRAPHITE_SERVER,
STATSD_PORT)
sock.close
Because we use statsd and graphite for high-volume event and stats monitoring, we were effectively triggering numerous calls getaddrinfo on every API call, and potentially tens of thousands every minute.
I modified this code to use the internal IP address, not the DNS name, of our graphite server, and was able to resolve the latency issue (presumably because the internal AWS VPC DNS server was not equipped to handle such a high volume of requests).
Now that my issue is resolved, I would love to know why the UDP implementation in Ruby is not using a cached IP address value (presumably based on the TTL of the domain name entry). Here is the relevant line and the function in full, you can see the call to rsock_addrinfo just at the end:
static VALUE
udp_send(int argc, VALUE *argv, VALUE sock)
{
VALUE flags, host, port;
struct udp_send_arg arg;
VALUE ret;
if (argc == 2 || argc == 3) {
return rsock_bsock_send(argc, argv, sock);
}
rb_scan_args(argc, argv, "4", &arg.sarg.mesg, &flags, &host, &port);
StringValue(arg.sarg.mesg);
GetOpenFile(sock, arg.fptr);
arg.sarg.fd = arg.fptr->fd;
arg.sarg.flags = NUM2INT(flags);
arg.res = rsock_addrinfo(host, port, rsock_fd_family(arg.fptr->fd), SOCK_DGRAM, 0);
ret = rb_ensure(udp_send_internal, (VALUE)&arg,
rsock_freeaddrinfo, (VALUE)arg.res);
if (!ret) rsock_sys_fail_host_port("sendto(2)", host, port);
return ret;
}
I assume this decision is intentional and would love to learn more about the reasons why.
getaddrinfo does not return data about the TTL... because it may not have it at all in fact, as the resolution may not necessarily be done over the DNS (could be hosts file, LDAP, etc. see /etc/nsswitch.conf)
From its manual here is the structure returned:
int getaddrinfo(const char *hostname, const char *servname, const struct addrinfo *hints, struct addrinfo **res);
struct addrinfo {
int ai_flags; /* input flags */
int ai_family; /* protocol family for socket */
int ai_socktype; /* socket type */
int ai_protocol; /* protocol for socket */
socklen_t ai_addrlen; /* length of socket-address */
struct sockaddr *ai_addr; /* socket-address for socket */
char *ai_canonname; /* canonical name for service location */
struct addrinfo *ai_next; /* pointer to next in list */
};
After a successful call to getaddrinfo(), *res is a pointer to a linked list of one or more addrinfo structures.
So it is up to the thing "behind" getaddrinfo to do some caching or not, because getaddrinfo may have used the DNS to retrieve data, or not.
Some specific API for DNS, like getdnsapi will give back to the caller some information on the TTL, see https://getdnsapi.net/documentation/spec/ and example 6.2
6·2 Get IPv4 and IPv6 Addresses for a Domain Name
This example is similar to the previous one, except that it retrieves more information than just the addresses, so it traverses the replies_tree. In this case, it gets both the addresses and their TTLs.
Without any cache layer anywhere, since UDP is stateless, any new send must trigger resolution in some way or form.
You said:
"modified this code to use the internal IP address, not the DNS name"
You should instead install a local (on the box) recursive caching nameserver, such as unbound. All your local applications will benefit from it, and a faster DNS resolution (depending on how /etc/nsswitch.conf, /etc/resolv.conf and /etc/hosts are setup also).
For the associated bug report hinted by #Casper it seems at its core more an issue about IPv6 vs IPv4 which could be solved either by adjusting /etc/gai.conf or equivalent or doing some more clever programming around opening the connection, with the so called "happy eyeball algorithm" where you try to resolve both A and AAAA at the same time which means two parallel DNS queries (because you can not combine them into one per the protocol) and try to use the fastest one coming back, with a slight preference for AAAA if you want to be in the modern camp so you would fire the A one only some given amount of milliseconds after the AAAA to catch the case where you do not get a reply at all for AAAA or a negative one. See RFC6555 for details.
I am writing a PCI driver with a character device for an interface (Linux 4.9.13). Here's the scenario that bothers me:
Run touch /dev/foo0 which creates a normal file in the /dev directory.
Load the driver module. Here's a pseudo code representing what happens there (pretty standard character device registration):
// When the module is initialized:
alloc_chrdev_region(&dev, 0, 256, "foo");
class = class_create(THIS_MODULE, "foo");
// Later, when a suitable PCI device is connected the probe function
// calls the following functions:
cdev_init(dev->md_cdev, &fops);
dev->md_devnum = MKDEV(major, 0 + index);
res = cdev_add(dev->md_cdev, dev->md_devnum, 1);
dev->md_sysfsdev = device_create(class, 0, dev->md_devnum, 0, "foo%d", index);
Details:
index is just another free index
What seems weird to me is nothing raises an error that there is already a /dev/foo0 file which is not a character device. I do check all the errors (I think so) but I omitted related code for the sake of conciseness. Everything works as expected if I do not run touch /dev/foo0. Otherwise, I can neither read nor write to the device.
Why is it so? Shouldn't device_create return an error or at least create /dev/foo1 instead?
I am using an Arduino Uno with the Desloo W5100 Ethernet shield. Whenever I try to make calls to Parse using Temboo, the device just hangs. Sometimes for minutes...sometimes indefinitely. Here is what I run:
void updateParseDoorState() {
if (!ENABLE_DOOR_STATE_PUSHES) {
Serial.println("Door state pushing disabled. Skipping.");
return;
}
Serial.println("Pushing door state to database...");
TembooChoreo UpdateObjectChoreo(client);
// Invoke the Temboo client
UpdateObjectChoreo.begin();
// Set Temboo account credentials
UpdateObjectChoreo.setAccountName(TEMBOO_ACCOUNT);
UpdateObjectChoreo.setAppKeyName(TEMBOO_APP_KEY_NAME);
UpdateObjectChoreo.setAppKey(TEMBOO_APP_KEY);
// Set profile to use for execution
UpdateObjectChoreo.setProfile("ParseAccount");
// Set Choreo inputs
String ObjectIDValue = "xxxxxxxxxx";
UpdateObjectChoreo.addInput("ObjectID", ObjectIDValue);
String ClassNameValue = "DoorState";
UpdateObjectChoreo.addInput("ClassName", ClassNameValue);
String ObjectContentsValue = (currentState == OPEN) ? "{\"isOpen\":true}" : "{\"isOpen\":false}";
UpdateObjectChoreo.addInput("ObjectContents", ObjectContentsValue);
// Identify the Choreo to run
UpdateObjectChoreo.setChoreo("/Library/Parse/Objects/UpdateObject");
// Run the Choreo; when results are available, print them to serial
int returnStatus = UpdateObjectChoreo.run();
if (returnStatus != 0){
setEthernetIndicator(EthernetStatus::SERVICES_DISCONNECTED);
Serial.print("Temboo error: "); Serial.println(returnStatus);
// read the name of the next output item
String returnResultName = UpdateObjectChoreo.readStringUntil('\x1F');
returnResultName.trim(); // use “trim” to get rid of newlines
Serial.print("Return result name: "); Serial.println(returnResultName);
// read the value of the next output item
String returnResultData = UpdateObjectChoreo.readStringUntil('\x1E');
returnResultData.trim(); // use “trim” to get rid of newlines
Serial.print("Return result data: "); Serial.println(returnResultData);
}
/*while(UpdateObjectChoreo.available()) {
char c = UpdateObjectChoreo.read();
Serial.print(c);
}*/
UpdateObjectChoreo.close();
Serial.println("Pushed door state to database!");
Serial.println("Waiting 30s to avoid overloading Temboo...");
delay(30000);
}
I get this in the serial monitor:
Current state:6666ÿ &‰ SP S P U WR SR R PR P 66Temboo error: 223
This indicates that there is some type of HTTP error, but I never get to print what the error is...because the serial monitor is stuck there forever. And eventually disconnects.
I work at Temboo.
It sounds like you might be running out of memory on your board (a common occurrence on resource-constrained hardware like Arduino). You can find our tutorial on how to conserve memory usage while using Temboo here:
https://temboo.com/hardware/profiles
Feel free to get in touch with Temboo Support at any time if you have further questions - we're always available and happy to help.
http://hadoop.apache.org/docs/r2.1.0-beta/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html
I am try to make the example work well from the above link.but I can't compile the code below
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(512);
amContainer.setResource(capability);
// Set the container launch content into the
// ApplicationSubmissionContext
appContext.setAMContainerSpec(amContainer);
amContainer is ContainerLaunchContext and my hadoop version is 2.1.0-beta.
I did some investigation. I found there's no method "setResource" in ContainerLaunchContext
I have 3 question about this
1) the method has been removed or something?
2) if the method has been removed, how can I do now?
3) is there any doc about yarn, because I found the doc in website is very easy, I hope I can get a manual or something. for example,
capability.setMemory(512);
I don't know it's 512k or 512M according comments in code.
This is actually proper solution to the question. Previous answer might cause incorrect execution !!!
#Dyin I couldn't fit it in the comment ;) Validated for 2.2.0 and 2.3.0
Driver setting up resources for AppMaster:
ApplicationSubmissionContext appContext = app.getApplicationSubmissionContext();
ApplicationId appId = appContext.getApplicationId();
appContext.setApplicationName(this.appName);
// Set up the container launch context for the application master
ContainerLaunchContext amContainer = Records.newRecord(ContainerLaunchContext.class);
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(amMemory);
appContext.setResource(capability);
appContext.setAMContainerSpec(amContainer);
Priority pri = Records.newRecord(Priority.class);
pri.setPriority(amPriority);
appContext.setPriority(pri);
appContext.setQueue(amQueue);
// Submit the application to the applications manager
yarnClient.submitApplication(appContext); // this.yarnClient = YarnClient.createYarnClient();
In ApplicationMaster this is how you should specify resources for containers (workers).
private AMRMClient.ContainerRequest setupContainerAskForRM() {
// setup requirements for hosts
// using * as any host will do for the distributed shell app
// set the priority for the request
Priority pri = Records.newRecord(Priority.class);
pri.setPriority(requestPriority);
// Set up resource type requirements
// For now, only memory is supported so we set memory requirements
Resource capability = Records.newRecord(Resource.class);
capability.setMemory(containerMemory);
AMRMClient.ContainerRequest request = new AMRMClient.ContainerRequest(capability, null, null,
pri);
return request;
}
Some run() or main() method in your AppMaster
AMRMClientAsync.CallbackHandler allocListener = new RMCallbackHandler();
resourceManager = AMRMClientAsync.createAMRMClientAsync(1000, allocListener);
resourceManager.init(conf);
resourceManager.start();
for (int i = 0; i < numTotalContainers; ++i) {
AMRMClient.ContainerRequest containerAsk = setupContainerAskForRM();
resourceManager.addContainerRequest(containerAsk); //
}
Launching containers
You can use the original answer solution (java cmd), but it's just a cherry on top. It should work anyway.
You can set memory available to ApplicationMaster via commend. As such:
// Set the necessary command to execute the application master
Vector<CharSequence> vargs = new Vector<CharSequence>(30);
...
vargs.add("-Xmx" + amMemory + "m"); // notice "m" indicating megabytes, you can use also -Xms combined with -Xmx
... // transform vargs to String commands
amContainer.setCommands(commands);
This should solve your problem. As for the 3 questions. Yarn is rapidly evolving software. My advice forget documentation, get source code and read it. This will answer a lot of your questions.
what is the equivalent of msync [unix sys call] in windows? I am looking for MSDN api in c,C++ space.
More info on msync can be found at http://opengroup.org/onlinepubs/007908799/xsh/msync.html
FlushViewOfFile
Checkout the Python 2.6 mmapmodule.c for an example of FlushViewOfFile and msync in use:
/*
/ Author: Sam Rushing <rushing#nightmare.com>
/ Hacked for Unix by AMK
/ $Id: mmapmodule.c 65859 2008-08-19 17:47:13Z thomas.heller $
/ Modified to support mmap with offset - to map a 'window' of a file
/ Author: Yotam Medini yotamm#mellanox.co.il
/
/ mmapmodule.cpp -- map a view of a file into memory
/
/ todo: need permission flags, perhaps a 'chsize' analog
/ not all functions check range yet!!!
/
/
/ This version of mmapmodule.c has been changed significantly
/ from the original mmapfile.c on which it was based.
/ The original version of mmapfile is maintained by Sam at
/ ftp://squirl.nightmare.com/pub/python/python-ext.
*/
static PyObject *
mmap_flush_method(mmap_object *self, PyObject *args)
{
Py_ssize_t offset = 0;
Py_ssize_t size = self->size;
CHECK_VALID(NULL);
if (!PyArg_ParseTuple(args, "|nn:flush", &offset, &size))
return NULL;
if ((size_t)(offset + size) > self->size) {
PyErr_SetString(PyExc_ValueError, "flush values out of range");
return NULL;
}
#ifdef MS_WINDOWS
return PyInt_FromLong((long) FlushViewOfFile(self->data+offset, size));
#elif defined(UNIX)
/* XXX semantics of return value? */
/* XXX flags for msync? */
if (-1 == msync(self->data + offset, size, MS_SYNC)) {
PyErr_SetFromErrno(mmap_module_error);
return NULL;
}
return PyInt_FromLong(0);
#else
PyErr_SetString(PyExc_ValueError, "flush not supported on this system");
return NULL;
#endif
}
UPDATE:
I don't think you are going to find complete parity in the win32 mapped file APIs. The FlushViewOfFile API doesn't have a synchronous flavor (probably because of the possible impact of the cache manager). If precise control over when data is written to disk is required perhaps you can use the FILE_FLAG_NO_BUFFERING and FILE_FLAG_WRITE_THROUGH flags with the CreateFile API when you create the handle to your mapped file?
Windows equivalent for flushing all filemapping is
void FlushToHardDrive(LPVOID fileMapAddress,HANDLE hFile)
{
FlushViewOfFile(fileMappAddress,0); //Async flush of dirty pages
FlushFileBuffers(hFiles); // flush metadata and wait
}
And for flushing part of filemapping
void FlushToHardDrive(LPVOID address,DWORD size, HANDLE hFile)
{
FlushViewOfFile(address,size); //Async flush of region
FlushFileBuffers(hFiles); // flush metadata and wait
}
This is described in MSDN here
Flag FILE_FLAG_NO_BUFFERING actually do nothing for memory mapped files (described here), also even file handle is created with these flags, metadata of file can be cached and not flushed, so FlushFileBuffers is always required for IO and MM, if you want to be completely sure that all data (including file access time) is saved. This behavior is described here
P.S. Some real world example: SQLite use MM-files almost only for reading, so when you are using MM-files for write/update you need understand all side effects for this scenario
I suspect FlushViewOfFile actually is the right thing. When I read the man page for msync, I would not assume that it is actually flushing the disk cache (the cache in the disk unit, as opposed to the system cache in main memory).
FlushViewOfFile won't return until the disk stack has completed the writes; like the msync documentation, it says nothing about what happens in the disk cache. We should probably take a look at making that more clear in the documentation.