Why is the Java 7 Bytecode Verifier complaining about this Stack Frame? - bytecode

I have a method which I have altered in a Java 7 (major version 51) class. Using javap, I've looked at the bytecode and the Stack Frame Map. Everything looks fine:
public int addOne(int);
flags: ACC_PUBLIC
Code:
stack=2, locals=2, args_size=2
0: iload_1
1: iconst_0
2: invokestatic #50 // Method isSomething:(I)Z
5: ifeq 12
8: iconst_0
9: goto 13
12: iconst_1
13: iadd
14: ireturn
StackMapTable: number_of_entries = 2
frame_type = 255 /* full_frame */
offset_delta = 12
locals = [ class test/Target, int ]
stack = [ int ]
frame_type = 255 /* full_frame */
offset_delta = 0
locals = [ class test/Target, int ]
stack = [ int, int ]
This verifier throws this exception:
java.lang.VerifyError: Expecting a stackmap frame at branch target 12
Exception Details:
Location:
test/Target.addOne(I)I #5: ifeq
Reason:
Expected stackmap frame at this location.
Bytecode:
0000000: 1b03 b800 3299 0007 03a7 0004 0460 ac
What's driving me crazy is that I had the compiler generate the same code from Java source, and it looks like this:
public int addOne(int);
flags: ACC_PUBLIC
Code:
stack=2, locals=2, args_size=2
0: iload_1
1: iconst_0
2: invokestatic #16 // Method isSomething:(I)Z
5: ifeq 12
8: iconst_0
9: goto 13
12: iconst_1
13: iadd
14: ireturn
StackMapTable: number_of_entries = 2
frame_type = 76 /* same_locals_1_stack_item */
stack = [ int ]
frame_type = 255 /* full_frame */
offset_delta = 0
locals = [ class test/Target, int ]
stack = [ int, int ]
Notice that the only difference in the stack frame map is that the synthetic map has all full frames -- but that shouldn't cause a difference. Does anyone know why the verfier might not like my synthetic map?

I am unable to reproduce this problem. Perhaps you are creating the stack frames in a way that javap will still read but isn't actually valid? Because my edited class has the same javap output but it verifies just fine. If you post the actual classfile I can see if I can find the problem, because I don't think there's anything further I can do with just the Javap output.
Source:
public class ArrayTest {
public int addOne(int x){
return x + (isSomething(0) ? 0 : 1);
}
public static boolean isSomething(int z) {return true;}
}
Javap output of method in original class
public int addOne(int);
flags: ACC_PUBLIC
Code:
stack=2, locals=2, args_size=2
0: iload_1
1: iconst_0
2: invokestatic #2 // Method isSomething:(I)Z
5: ifeq 12
8: iconst_0
9: goto 13
12: iconst_1
13: iadd
14: ireturn
StackMapTable: number_of_entries = 2
frame_type = 76 /* same_locals_1_stack_item */
stack = [ int ]
frame_type = 255 /* full_frame */
offset_delta = 0
locals = [ class ArrayTest, int ]
stack = [ int, int ]
Javap output of method in edited class
public int addOne(int);
flags: ACC_PUBLIC
Code:
stack=2, locals=2, args_size=2
0: iload_1
1: iconst_0
2: invokestatic #15 // Method ArrayTest.isSomething:(
)Z
5: ifeq 12
8: iconst_0
9: goto 13
12: iconst_1
13: iadd
14: ireturn
StackMapTable: number_of_entries = 2
frame_type = 255 /* full_frame */
offset_delta = 12
locals = [ class ArrayTest2, int ]
stack = [ int ]
frame_type = 255 /* full_frame */
offset_delta = 0
locals = [ class ArrayTest2, int ]
stack = [ int, int ]
As you can see, I have the same Javap output, but my class works just fine.

The answer is that javassist sucks and I am deeply regretting using it.
The StackMapTable attribute is gotten from a call to CodeAttribute.getAttribute(String tag). Even though this is how you access it, there is no API to add it back, unless it is of type StackMapTable. The only API that accepts a vanilla AttributeInfo as a parameter is on the MethodInfo class.
In cases where the method did not need (or have) a stack frame already, you get a null. If you create an AttributeInfo structure for a new stack frame map, you shouldn't add the to the MethodInfo (where the addAttribute API is), but the CodeAttribute where it belongs.
This is what I was doing:
MethodInfo mi ...
AttributeInfo attr ...
mi.addAttribute(attr);
This is what I needed to do:
CodeAttribute ca ...
ca.getAttributes().add(attr);
(Of course, ca.getAttributes()returns an untyped List because we all miss 2004.)
I dug into the method that allows you to add a type StackFramMap to the CodeAttribute and figured out this work around for the lack of a generic API.
The result of using the top construct is that javap will make it appear that you have a proper StackMapTable. You do, but it's attached to the wrong object and you can't see that from the javap output.
I didn't use ASM for my project because I found its obsessive use of the Visitor Pattern to be annoying. I now admit that this was a bad decision. Since javassist hasn't had an update since 2012, I'm wondering if the project is dead. I certainly have a truckload of revisions I'd push. It is a mess.
EDIT Oh wow. Javassist internal code assumes that any StackMapTable attribute it its own internal StackMapTable type (because how else would you add the StackMapTable attribute). I guess I could create my own StackMapTable instance, except the SFM constructors are package protected for no apparent reason. It just gets worse and worse...

http://www.csg.ci.i.u-tokyo.ac.jp/~chiba/javassist/html/javassist/bytecode/MethodInfo.html
Now you can call:
mi.rebuildStackMapIf6(pool, cf);
It will rebuild stack map, and don't use or need -XX:-UseSplitVerifier

Related

Theano Import Error-Windows 7-64b

I just installed the Theano.
I have a problem in importing theano in python. When I import Theano in the python 27 32 bit, in Windows 7 64 bit, I get the following errors and warning: I also should add that currently I have installed GCC 4.8.1. What I have to do in order to fix it.
Thanks, Afshin
===============================
00001 #include <Python.h>
00002 #include "structmember.h"
00003 #include <sys/time.h>
00004
00005 // Old Python compatibility from here:
00006 // http://www.python.org/dev/peps/pep-0353/
00007 #if PY_VERSION_HEX < 0x02050000 && !defined(PY_SSIZE_T_MIN)
00008 typedef int Py_ssize_t;
00009 #define PY_SSIZE_T_MAX INT_MAX
00010 #define PY_SSIZE_T_MIN INT_MIN
00011 // This one was taken from:
00012 // http://svn.python.org/projects/python/trunk/Modules/_ctypes/ctypes.h
00013 #define PyNumber_AsSsize_t(ob, exc) PyInt_AsLong(ob)
00014 #endif
00015
00016 #if PY_VERSION_HEX >= 0x03000000
00017 #include "numpy/npy_3kcompat.h"
00018 #define PyCObject_AsVoidPtr NpyCapsule_AsVoidPtr
00019 #define PyCObject_GetDesc NpyCapsule_GetDesc
00020 #define PyCObject_Check NpyCapsule_Check
00021 #endif
00022
00023 #ifndef Py_TYPE
00024 #define Py_TYPE(obj) obj->ob_type
00025 #endif
00026
00027 /**
00028
00029 TODO:
00030 - Check max supported depth of recursion
00031 - CLazyLinker should add context information to errors caught during evaluation. Say what node we were on, add the traceback attached to the node.
00032 - Clear containers of fully-useed intermediate results if allow_gc is 1
00033 - Add timers for profiling
00034 - Add support for profiling space used.
00035
00036
00037 */
00038 static double pytime(const struct timeval * tv)
00039 {
00040 struct timeval t;
00041 if (!tv)
00042 {
00043 tv = &t;
00044 gettimeofday(&t, NULL);
00045 }
00046 return (double) tv->tv_sec + (double) tv->tv_usec / 1000000.0;
00047 }
00048
00049 /**
00050 Helper routine to convert a PyList of integers to a c array of integers.
00051 */
00052 static int unpack_list_of_ssize_t(PyObject * pylist, Py_ssize_t **dst, Py_ssize_t *len,
00053 const char* kwname)
00054 {
00055 Py_ssize_t buflen, *buf;
00056 if (!PyList_Check(pylist))
00057 {
00058 PyErr_Format(PyExc_TypeError, "%s must be list", kwname);
00059 return -1;
00060 }
00061 assert (NULL == *dst);
00062 *len = buflen = PyList_Size(pylist);
00063 *dst = buf = (Py_ssize_t*)calloc(buflen, sizeof(Py_ssize_t));
00064 assert(buf);
00065 for (int ii = 0; ii < buflen; ++ii)
00066 {
00067 PyObject * el_i = PyList_GetItem(pylist, ii);
00068 Py_ssize_t n_i = PyNumber_AsSsize_t(el_i, PyExc_IndexError);
00069 if (PyErr_Occurred())
00070 {
00071 free(buf);
00072 *dst = NULL;
00073 return -1;
00074 }
00075 buf[ii] = n_i;
00076 }
00077 return 0;
00078 }
00079
00080 /**
00081
00082 CLazyLinker
00083
00084
00085 */
00086 typedef struct {
00087 PyObject_HEAD
00088 /* Type-specific fields go here. */
00089 PyObject * nodes; // the python list of nodes
00090 PyObject * thunks; // python list of thunks
00091 PyObject * pre_call_clear; //list of cells to clear on call.
00092 int allow_gc;
00093 Py_ssize_t n_applies;
00094 int n_vars; // number of variables in the graph
00095 int * var_computed; // 1 or 0 for every variable
00096 PyObject ** var_computed_cells;
00097 PyObject ** var_value_cells;
00098 Py_ssize_t **dependencies; // list of vars dependencies for GC
00099 Py_ssize_t *n_dependencies;
00100
00101 Py_ssize_t n_output_vars;
00102 Py_ssize_t * output_vars; // variables that *must* be evaluated by call
00103
00104 int * is_lazy; // 1 or 0 for every thunk
00105
00106 Py_ssize_t * var_owner; // nodes[[var_owner[var_idx]]] is var[var_idx]->owner
00107 int * var_has_owner; // 1 or 0
00108
00109 Py_ssize_t * node_n_inputs;
00110 Py_ssize_t * node_n_outputs;
00111 Py_ssize_t ** node_inputs;
00112 Py_ssize_t ** node_outputs;
00113 Py_ssize_t * node_inputs_outputs_base; // node_inputs and node_outputs point into this
00114 Py_ssize_t * node_n_prereqs;
00115 Py_ssize_t ** node_prereqs;
00116
00117 Py_ssize_t * update_storage; // input cells to update with the last outputs in output_vars
00118 Py_ssize_t n_updates;
00119
00120 void ** thunk_cptr_fn;
00121 void ** thunk_cptr_data;
00122 PyObject * call_times;
00123 PyObject * call_counts;
00124 int do_timing;
00125 int need_update_inputs;
00126 int position_of_error; // -1 for no error, otw the index into `thunks` that failed.
00127 } CLazyLinker;
00128
00129
00130 static void
00131 CLazyLinker_dealloc(PyObject* _self)
00132 {
00133 CLazyLinker* self = (CLazyLinker *) _self;
00134 free(self->thunk_cptr_fn);
00135 free(self->thunk_cptr_data);
00136
00137 free(self->is_lazy);
00138
00139 free(self->update_storage);
00140
00141 if (self->node_n_prereqs)
00142 {
00143 for (int i = 0; i < self->n_applies; ++i)
00144 {
00145 free(self->node_prereqs[i]);
00146 }
00147 }
00148 free(self->node_n_prereqs);
00149 free(self->node_prereqs);
00150 free(self->node_inputs_outputs_base);
00151 free(self->node_n_inputs);
00152 free(self->node_n_outputs);
00153 free(self->node_inputs);
00154 free(self->node_outputs);
00155
00156 if (self->dependencies)
00157 {
00158 for (int i = 0; i < self->n_vars; ++i)
00159 {
00160 free(self->dependencies[i]);
00161 }
00162 free(self->dependencies);
00163 free(self->n_dependencies);
00164 }
00165
00166 free(self->var_owner);
00167 free(self->var_has_owner);
00168 free(self->var_computed);
00169 if (self->var_computed_cells)
00170 {
00171 for (int i = 0; i < self->n_vars; ++i)
00172 {
00173 Py_DECREF(self->var_computed_cells[i]);
00174 Py_DECREF(self->var_value_cells[i]);
00175 }
00176 }
00177 free(self->var_computed_cells);
00178 free(self->var_value_cells);
00179 free(self->output_vars);
00180
00181 Py_XDECREF(self->nodes);
00182 Py_XDECREF(self->thunks);
00183 Py_XDECREF(self->call_times);
00184 Py_XDECREF(self->call_counts);
00185 Py_XDECREF(self->pre_call_clear);
00186 Py_TYPE(self)->tp_free((PyObject*)self);
00187 }
00188 static PyObject *
00189 CLazyLinker_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
00190 {
00191 CLazyLinker *self;
00192
00193 self = (CLazyLinker *)type->tp_alloc(type, 0);
00194 if (self != NULL) {
00195 self->nodes = NULL;
00196 self->thunks = NULL;
00197 self->pre_call_clear = NULL;
00198
00199 self->allow_gc = 1;
00200 self->n_applies = 0;
00201 self->n_vars = 0;
00202 self->var_computed = NULL;
00203 self->var_computed_cells = NULL;
00204 self->var_value_cells = NULL;
00205 self->dependencies = NULL;
00206 self->n_dependencies = NULL;
00207
00208 self->n_output_vars = 0;
00209 self->output_vars = NULL;
00210
00211 self->is_lazy = NULL;
00212
......
(I just removed some codes, because Stachoverflow does not allow more than 30000 character......
......
01075 PyModule_AddObject(m, "CLazyLinker", (PyObject *)&lazylinker_ext_CLazyLinkerType);
01076
01077 return RETVAL;
01078 }
01079
01080
Problem occurred during compilation with the command line below:
C:\cygwin64\bin\g++.exe -shared -g -march=nehalem -mmmx -mno-3dnow -msse -msse2 -msse3 -mssse3 -mno-sse4a -mcx16 -msahf -mno-movbe -mno-aes -mno-sha -mno-pclmul -mpopcnt -mno-abm -mno-lwp -mno-fma -mno-fma4 -mno-xop -mno-bmi -mno-bmi2 -mno-tbm -mno-avx -mno-avx2 -msse4.2 -msse4.1 -mno-lzcnt -mno-rtm -mno-hle -mno-rdrnd -mno-f16c -mno-fsgsbase -mno-rdseed -mno-prfchw -mno-adx -mfxsr -mno-xsave -mno-xsaveopt -mno-avx512f -mno-avx512er -mno-avx512cd -mno-avx512pf -mno-prefetchwt1 --param l1-cache-size=32 --param l1-cache-line-size=64 --param l2-cache-size=6144 -mtune=nehalem -D NPY_NO_DEPRECATED_API=NPY_1_7_API_VERSION -m32 -IC:\python2732\lib\site-packages\numpy-1.9.0-py2.7-win32.egg\numpy\core\include -IC:\python2732\include -o C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\lazylinker_ext.pyd C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp -LC:\python2732\libs -LC:\python2732 -lpython27
===============================
In file included from C:\python2732\include/Python.h:86:0,
from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:
C:\python2732\include/intobject.h:46:35: error: expected initializer before 'PyInt_AsUnsignedLongLongMask'
PyAPI_FUNC(unsigned PY_LONG_LONG) PyInt_AsUnsignedLongLongMask(PyObject *);
^
In file included from C:\python2732\include/Python.h:8:0,
from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:
C:\python2732\include/pyconfig.h:302:23: error: '__int64' was not declared in this scope
# define PY_LONG_LONG __int64
^
C:\python2732\include/longobject.h:50:44: note: in expansion of macro 'PY_LONG_LONG'
PyAPI_FUNC(PyObject *) PyLong_FromLongLong(PY_LONG_LONG);
^
In file included from C:\python2732\include/Python.h:58:0,
from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:
C:\python2732\include/pyconfig.h:302:23: error: '__int64' does not name a type
# define PY_LONG_LONG __int64
^
C:\python2732\include/pyport.h:793:34: note: in definition of macro 'PyAPI_FUNC'
# define PyAPI_FUNC(RTYPE) RTYPE
^
C:\python2732\include/longobject.h:52:12: note: in expansion of macro 'PY_LONG_LONG'
PyAPI_FUNC(PY_LONG_LONG) PyLong_AsLongLong(PyObject *);
^
In file included from C:\python2732\include/Python.h:88:0,
from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:
C:\python2732\include/longobject.h:53:35: error: expected initializer before 'PyLong_AsUnsignedLongLong'
PyAPI_FUNC(unsigned PY_LONG_LONG) PyLong_AsUnsignedLongLong(PyObject *);
^
C:\python2732\include/longobject.h:54:35: error: expected initializer before 'PyLong_AsUnsignedLongLongMask'
PyAPI_FUNC(unsigned PY_LONG_LONG) PyLong_AsUnsignedLongLongMask(PyObject *);
^
In file included from C:\python2732\include/Python.h:58:0,
from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:
C:\python2732\include/pyconfig.h:302:23: error: '__int64' does not name a type
# define PY_LONG_LONG __int64
^
C:\python2732\include/pyport.h:793:34: note: in definition of macro 'PyAPI_FUNC'
# define PyAPI_FUNC(RTYPE) RTYPE
^
C:\python2732\include/longobject.h:55:12: note: in expansion of macro 'PY_LONG_LONG'
PyAPI_FUNC(PY_LONG_LONG) PyLong_AsLongLongAndOverflow(PyObject *, int *);
^
Traceback (most recent call last):
File "C:\Users\Dell\Documents\Pyton\Home Work 1\test-theano\test-theano.py", line 7, in <module>
from theano import *
File "C:\python2732\lib\site-packages\theano-0.7.0-py2.7.egg\theano\__init__.py", line 55, in <module>
from theano.compile import \
File "C:\python2732\lib\site-packages\theano-0.7.0-py2.7.egg\theano\compile\__init__.py", line 9, in <module>
from theano.compile.function_module import *
File "C:\python2732\lib\site-packages\theano-0.7.0-py2.7.egg\theano\compile\function_module.py", line 18, in <module>
import theano.compile.mode
File "C:\python2732\lib\site-packages\theano-0.7.0-py2.7.egg\theano\compile\mode.py", line 11, in <module>
import theano.gof.vm
File "C:\python2732\lib\site-packages\theano-0.7.0-py2.7.egg\theano\gof\vm.py", line 568, in <module>
import lazylinker_c
File "C:\python2732\lib\site-packages\theano-0.7.0-py2.7.egg\theano\gof\lazylinker_c.py", line 116, in <module>
preargs=args)
File "C:\python2732\lib\site-packages\theano-0.7.0-py2.7.egg\theano\gof\cmodule.py", line 2010, in compile_str
(status, compile_stderr.replace('\n', '. ')))
Exception: Compilation failed (return status=1): In file included from C:\python2732\include/Python.h:86:0,. from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:. C:\python2732\include/intobject.h:46:35: error: expected initializer before 'PyInt_AsUnsignedLongLongMask'. PyAPI_FUNC(unsigned PY_LONG_LONG) PyInt_AsUnsignedLongLongMask(PyObject *);. ^. In file included from C:\python2732\include/Python.h:8:0,. from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:. C:\python2732\include/pyconfig.h:302:23: error: '__int64' was not declared in this scope. # define PY_LONG_LONG __int64. ^. C:\python2732\include/longobject.h:50:44: note: in expansion of macro 'PY_LONG_LONG'. PyAPI_FUNC(PyObject *) PyLong_FromLongLong(PY_LONG_LONG);. ^. In file included from C:\python2732\include/Python.h:58:0,. from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:. C:\python2732\include/pyconfig.h:302:23: error: '__int64' does not name a type. # define PY_LONG_LONG __int64. ^. C:\python2732\include/pyport.h:793:34: note: in definition of macro 'PyAPI_FUNC'. # define PyAPI_FUNC(RTYPE) RTYPE. ^. C:\python2732\include/longobject.h:52:12: note: in expansion of macro 'PY_LONG_LONG'. PyAPI_FUNC(PY_LONG_LONG) PyLong_AsLongLong(PyObject *);. ^. In file included from C:\python2732\include/Python.h:88:0,. from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:. C:\python2732\include/longobject.h:53:35: error: expected initializer before 'PyLong_AsUnsignedLongLong'. PyAPI_FUNC(unsigned PY_LONG_LONG) PyLong_AsUnsignedLongLong(PyObject *);. ^. C:\python2732\include/longobject.h:54:35: error: expected initializer before 'PyLong_AsUnsignedLongLongMask'. PyAPI_FUNC(unsigned PY_LONG_LONG) PyLong_AsUnsignedLongLongMask(PyObject *);. ^. In file included from C:\python2732\include/Python.h:58:0,. from C:\Users\Dell\AppData\Local\Theano\compiledir_Windows-7-6.1.7601-SP1-Intel64_Family_6_Model_30_Stepping_5_GenuineIntel-2.7.8-32\lazylinker_ext\mod.cpp:1:. C:\python2732\include/pyconfig.h:302:23: error: '__int64' does not name a type. # define PY_LONG_LONG __int64. ^. C:\python2732\include/pyport.h:793:34: note: in definition of macro 'PyAPI_FUNC'. # define PyAPI_FUNC(RTYPE) RTYPE. ^. C:\python2732\include/longobject.h:55:12: note: in expansion of macro 'PY_LONG_LONG'. PyAPI_FUNC(PY_LONG_LONG) PyLong_AsLongLongAndOverflow(PyObject *, int *);. ^.
This was also discussed on the Theano mailing list. For the benefit of StackOverflow users, the answer was to install and use MinGW in preference to Cygwin.
If both are installed, MinGW should be before Cygwin in the PATH.

Kernel oops when executing function to read hardware registers

I'm referencing this answer for crash help in analyzing this bit of code which caused problems. The context for everyone, I'm working a character driver, which will act as a pass through from user space directly to the hardware, for the ahci driver. I'm modifying the ahci driver accordingly for this purpose.
I'm starting small. I want to peek at the port registers for the HBA port 0 of the AHCI HBA on my VM. My character driver ioctl code:
switch (cmd) {
case AHCIP_GPORT_REG:
pPciDev = pci_get_device(0x8086, 0x2829, NULL);
if (pPciDev) {
/* This will set ret to the value that it needs to be. This
* is true of __put_user() too */
if ((ret = __get_user(off, (u32*)obj))) {
printk(KERN_INFO "unable to read from user space\n");
goto ioctl_quick_out;
}
reg = get_port_reg(&pPciDev->dev, off);
if ((ret = __put_user(reg, (u32*)obj)))
{
printk(KERN_INFO "Unable to write to user space\n");
}
pci_dev_put(pPciDev);
}
// This break wasn't in the code when it crashed
break;
default:
// POSIX compliance with this one (REF of LDD3)
ret = -ENOTTY;
}
The code from my modified version of ahci.c which this character driver calls into:
u32 get_port_reg(struct device *dev, u32 off)
{
struct Scsi_Host *shost = class_to_shost(dev);
struct ata_port *ap = ata_shost_to_port(shost);
void __iomem *port_mmio = ahci_port_base(ap);
return ioread32(port_mmio + off);
}
EXPORT_SYMBOL(get_port_reg);
The kernel oops that this caused, happened here:
PID: 3357 TASK: ffff88011c9b7500 CPU: 0 COMMAND: "peek"
#0 [ffff8800abfc79f0] machine_kexec at ffffffff8103b5bb
#1 [ffff8800abfc7a50] crash_kexec at ffffffff810c9852
#2 [ffff8800abfc7b20] oops_end at ffffffff8152e0f0
#3 [ffff8800abfc7b50] no_context at ffffffff8104c80b
#4 [ffff8800abfc7ba0] __bad_area_nosemaphore at ffffffff8104ca95
#5 [ffff8800abfc7bf0] bad_area at ffffffff8104cbbe
#6 [ffff8800abfc7c20] __do_page_fault at ffffffff8104d36f
#7 [ffff8800abfc7d40] do_page_fault at ffffffff8153003e
#8 [ffff8800abfc7d70] page_fault at ffffffff8152d3f5
[exception RIP: get_port_reg+18]
RIP: ffffffffa03c4cd2 RSP: ffff8800abfc7e28 RFLAGS: 00010246
RAX: 0000000000020101 RBX: 00007fff17273960 RCX: ffffffff812b0710
RDX: ffff88011ddd5000 RSI: 0000000000000000 RDI: ffff88011ddd5090
RBP: ffff8800abfc7e28 R8: 0000000000000000 R9: 0000000000000000
R10: 00000000000007d5 R11: 0000000000000006 R12: ffff88011ddd5000
R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
ORIG_RAX: ffffffffffffffff CS: 0010 SS: 0018
As you can see, the instruction pointer was get_port_reg+18. Since this function is quite small, here's the full disassembly
crash> dis get_port_reg
0xffffffffa03c4cc0 <get_port_reg>: push %rbp
0xffffffffa03c4cc1 <get_port_reg+1>: mov %rsp,%rbp
0xffffffffa03c4cc4 <get_port_reg+4>: nopl 0x0(%rax,%rax,1)
0xffffffffa03c4cc9 <get_port_reg+9>: mov 0x240(%rdi),%rax
0xffffffffa03c4cd0 <get_port_reg+16>: mov %esi,%esi
0xffffffffa03c4cd2 <get_port_reg+18>: mov 0x2838(%rax),%rdx
0xffffffffa03c4cd9 <get_port_reg+25>: mov 0x28(%rax),%eax
0xffffffffa03c4cdc <get_port_reg+28>: mov 0x10(%rdx),%rdx
0xffffffffa03c4ce0 <get_port_reg+32>: shl $0x7,%eax
0xffffffffa03c4ce3 <get_port_reg+35>: mov %eax,%eax
0xffffffffa03c4ce5 <get_port_reg+37>: add 0x28(%rdx),%rax
0xffffffffa03c4ce9 <get_port_reg+41>: lea 0x100(%rax,%rsi,1),%rdi
0xffffffffa03c4cf1 <get_port_reg+49>: callq 0xffffffff8129dde0 <ioread32>
0xffffffffa03c4cf6 <get_port_reg+54>: leaveq
0xffffffffa03c4cf7 <get_port_reg+55>: retq
0xffffffffa03c4cf8 <get_port_reg+56>: nopl 0x0(%rax,%rax,1)
As you might have guessed, I'm something of an assembly neophyte. Which line of code would be get_port_reg+18? I'm puzzled because I'm calling functions on each line of that function but the only call I see is to ioread32().
For reference, I've modeled my function get_port_reg after ahci_show_port_cmd() within the same file. I could not think of any other means of getting the struct pci_dev structure necessary on which this is to operate. Am I making bad use of get_pci_device() and pci_dev_put()? Is this not the issue at all?
Thanks for any help
Andy
I am going to post my own answer. The two commentators of my question have put me onto the correct path for fixing this. As I mentioned, my approach was to do something which I'd seen done elsewhere in the ahci driver (ahci.c). Basically, the assumption was simple, this function in ahci.c required a struct device* and from that was able to get the ata_port information that was required. I'd seen, in ahci.c, that the author had done struct device* = &pdev->dev; occasionally. In other words, I figured that the dev member of struct pci_dev was getting me what I needed. I was apparently unaware of "class types" or something similar (see #myaut's first comment). #alexhoppus essentially draws the same/similar conclusion based on the code and disassembly which I posted.
The fix which I have employed, and which does work nicely, is as follows:
/* ioctl code in character driver */
switch (cmd) {
case AHCIP_GPORT_REG:
pPciDev = pci_get_device(0x8086, 0x2829, NULL);
if (pPciDev) {
struct ata_host *pHost = NULL;
struct ata_port *pPort = NULL;
printk(KERN_INFO "found the PCI device\n");
/* Get the devices driver data */
pHost = pci_get_drvdata(pPciDev);
if (!pHost) {
ret = -EFAULT;
goto ioctl_valid_pci_dev_out;
}
/* for this test, we'll use just port 0 */
pPort = pHost->ports[0];
if (!pPort) {
ret = -EFAULT;
goto ioctl_valid_pci_dev_out;
}
/* This will set ret to the value that it needs to be. This
* is true of __put_user() too */
if ((ret = __get_user(off, (u32*)obj))) {
printk(KERN_INFO "unable to read from user space\n");
goto ioctl_valid_pci_dev_out;
}
reg = get_port_reg(pPort, off);
if ((ret = __put_user(reg, (u32*)obj)))
{
printk(KERN_INFO "Unable to write to user space\n");
}
}
break;
default:
// POSIX compliance with this one (REF of LDD3)
ret = -ENOTTY;
}
The ahci driver was modified thusly as well
u32 get_port_reg(struct ata_port* pPort, u32 off)
{
void __iomem *port_mmio = ahci_port_base(pPort);
return ioread32(port_mmio + off);
}
EXPORT_SYMBOL(get_port_reg);
Though this has fixed the issue for me, I would really appreciate someone explaining to me what is placed in (struct pci_dev)device.dev.p->driver_data. I can use, and have, the Linux cross referencing tools to see the data types. What is supposed to be stored instruct device_private`? This is the structure which I'm now using to get the data I need. I'd truly appreciate someone commenting on this answer to explain that one.
Thanks to #myaut and #alexhoppus

how can I get the `__NSAutoreleaseNoPool` address?

I tried with
extern void __NSAutoreleaseNoPool(void* obj);
but that results in an unresolved symbol when linking (not sure what Framework it needs, though).
I also tried
dlsym(RTLD_DEFAULT, "__NSAutoreleaseNoPool")
but that just gives be NULL.
And I tried with _dyld_lookup_and_bind and NSLookupSymbolInImage but they also don't work.
dsymutil and nm both find the symbol, though:
$ dsymutil -s --arch=x86_64
----------------------------------------------------------------------
/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
----------------------------------------------------------------------
Symbol table for: '/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation' (x86_64)
----------------------------------------------------------------------
Index n_strx n_type n_sect n_desc n_value
======== -------- ------------------ ------ ------ ----------------
[ 0] 00010795 1e (PEXT SECT ) 01 0000 0000000000000000 '__mh_dylib_header'
[ 1] 000107a7 0e ( SECT ) 01 0000 0000000000001c20 '+[NSObject(NSObject) load]'
[ 2] 000107c2 0e ( SECT ) 01 0000 0000000000002630 '___exceptionInit'
[ 3] 000107d3 0e ( SECT ) 01 0000 00000000000029e0 '___CFgetenv'
[ 4] 000107df 0e ( SECT ) 01 0000 0000000000002a50 '___CFBaseInitialize'
...
[ 1923] 0001e820 0e ( SECT ) 01 0000 000000000010ad30 '___NSAutoreleaseNoPool'
...
$ nm -arch x86_64 /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
...
000000000010ad30 t ___NSAutoreleaseNoPool
...
(That is on MacOSX 10.6. On later MacOSX versions, the symbol really does not seem to exists, at least I cannot find any ref via grep in /usr/lib and /System/Library/Frameworks and also LLDB does not find it. Probably it was removed somehow with ARC.)
So, how can I get that address in my code?
(Related questions: here and here)
(My motivation to do this is here.)
This works:
#include <dlfcn.h>
#include <stdio.h>
#import <Foundation/Foundation.h>
#include <mach-o/dyld.h>
#include <mach-o/nlist.h>
#include <string.h>
#include <assert.h>
// Adapted from:
// https://github.com/0xced/iOS-Artwork-Extractor/blob/master/Classes/FindSymbol.c
// Adapted from MoreAddrToSym / GetFunctionName()
// http://www.opensource.apple.com/source/openmpi/openmpi-8/openmpi/opal/mca/backtrace/darwin/MoreBacktrace/MoreDebugging/MoreAddrToSym.c
void *FindSymbol(const struct mach_header *img, const char *symbol)
{
if ((img == NULL) || (symbol == NULL))
return NULL;
// only 64bit supported
#if defined (__LP64__)
if(img->magic != MH_MAGIC_64)
// we currently only support Intel 64bit
return NULL;
struct mach_header_64 *image = (struct mach_header_64*) img;
struct segment_command_64 *seg_linkedit = NULL;
struct segment_command_64 *seg_text = NULL;
struct symtab_command *symtab = NULL;
unsigned int index;
struct load_command *cmd = (struct load_command*)(image + 1);
for (index = 0; index < image->ncmds; index += 1, cmd = (struct load_command*)((char*)cmd + cmd->cmdsize))
{
switch(cmd->cmd)
{
case LC_SEGMENT_64: {
struct segment_command_64* segcmd = (struct segment_command_64*)cmd;
if (!strcmp(segcmd->segname, SEG_TEXT))
seg_text = segcmd;
else if (!strcmp(segcmd->segname, SEG_LINKEDIT))
seg_linkedit = segcmd;
break;
}
case LC_SYMTAB:
symtab = (struct symtab_command*)cmd;
break;
default:
break;
}
}
if ((seg_text == NULL) || (seg_linkedit == NULL) || (symtab == NULL))
return NULL;
unsigned long vm_slide = (unsigned long)image - (unsigned long)seg_text->vmaddr;
unsigned long file_slide = ((unsigned long)seg_linkedit->vmaddr - (unsigned long)seg_text->vmaddr) - seg_linkedit->fileoff;
struct nlist_64 *symbase = (struct nlist_64*)((unsigned long)image + (symtab->symoff + file_slide));
char *strings = (char*)((unsigned long)image + (symtab->stroff + file_slide));
struct nlist_64 *sym;
for (index = 0, sym = symbase; index < symtab->nsyms; index += 1, sym += 1)
{
if (sym->n_un.n_strx != 0 && !strcmp(symbol, strings + sym->n_un.n_strx))
{
unsigned long address = vm_slide + sym->n_value;
if (sym->n_desc & N_ARM_THUMB_DEF)
return (void*)(address | 1);
else
return (void*)(address);
}
}
#endif
return NULL;
}
typedef void (*NSAutoreleaseNoPoolFunc) (void* obj);
void getNSAutoreleaseNoPool() {
const struct mach_header* img = NSAddImage("/System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation", NSADDIMAGE_OPTION_NONE);
NSAutoreleaseNoPoolFunc f = (NSAutoreleaseNoPoolFunc) FindSymbol((struct mach_header*)img, "___NSAutoreleaseNoPool");
printf("func: %p\n", f);
if(f) {
NSObject* foo = [[NSObject alloc] init];
f(foo);
}
}
It gets the same function pointer as within GDB.
Note that you wont see the common NSAutoreleaseNoPool log:
2014-02-18 14:46:26.583 a.out[24989:a0b] *** __NSAutoreleaseNoPool(): Object 0x7fff71154190 of class NSCFString autoreleased with no pool in place - just leaking
The standard backtrace, when that happens, is this:
(gdb) bt
#0 0x00007fff8724bd34 in __NSAutoreleaseNoPool ()
#1 0x00007fff87196e79 in _CFAutoreleasePoolAddObject ()
#2 0x00007fff87196be6 in -[NSObject(NSObject) autorelease] ()
The actual NSLog call is done in _CFAutoreleasePoolAddObject.
A note about __NSAutoreleaseNoPool, from Foundation/NSDebug.h:
/**************** Autorelease pool debugging ****************/
// Functions used as interesting breakpoints in a debugger
// void __NSAutoreleaseNoPool(void *object);
// Called to log the "Object X of class Y autoreleased with no
// pool in place - just leaking" message. If an environment
// variable named "NSAutoreleaseHaltOnNoPool" is set with string
// value "YES", the function will automatically break in the
// debugger (or terminate the process).
// void __NSAutoreleaseFreedObject(void *freedObject);
// Called when a previously freed object would be released
// by an autorelease pool. If an environment variable named
// "NSAutoreleaseHaltOnFreedObject" is set with string value
// "YES", the function will automatically break in the debugger
// (or terminate the process).
So, if you want to debug such cases, either start up GDB and issue b __NSAutoreleaseNoPool to setup the breakpoint on this function. Or do an export NSAutoreleaseHaltOnNoPool=1 in your shell.
__NSAutoreleaseNoPool is pretty simple:
(gdb) disassemble
Dump of assembler code for function __NSAutoreleaseNoPool:
0x00007fff8724bd30 <__NSAutoreleaseNoPool+0>: push %rbp
0x00007fff8724bd31 <__NSAutoreleaseNoPool+1>: mov %rsp,%rbp
0x00007fff8724bd34 <__NSAutoreleaseNoPool+4>: nop
0x00007fff8724bd35 <__NSAutoreleaseNoPool+5>: nopl 0x0(%rax)
0x00007fff8724bd39 <__NSAutoreleaseNoPool+9>: lea 0x2ced8(%rip),%rdi # 0x7fff87278c18 <__PRETTY_FUNCTION__.27904+480>
0x00007fff8724bd40 <__NSAutoreleaseNoPool+16>: callq 0x7fff871439e0 <__CFgetenv>
0x00007fff8724bd45 <__NSAutoreleaseNoPool+21>: test %rax,%rax
0x00007fff8724bd48 <__NSAutoreleaseNoPool+24>: je 0x7fff8724bd55 <__NSAutoreleaseNoPool+37>
0x00007fff8724bd4a <__NSAutoreleaseNoPool+26>: movzbl (%rax),%eax
0x00007fff8724bd4d <__NSAutoreleaseNoPool+29>: cmp $0x59,%al
0x00007fff8724bd4f <__NSAutoreleaseNoPool+31>: je 0x7fff8724bd60 <__NSAutoreleaseNoPool+48>
0x00007fff8724bd51 <__NSAutoreleaseNoPool+33>: cmp $0x79,%al
0x00007fff8724bd53 <__NSAutoreleaseNoPool+35>: je 0x7fff8724bd60 <__NSAutoreleaseNoPool+48>
0x00007fff8724bd55 <__NSAutoreleaseNoPool+37>: leaveq
0x00007fff8724bd56 <__NSAutoreleaseNoPool+38>: retq
0x00007fff8724bd57 <__NSAutoreleaseNoPool+39>: nopw 0x0(%rax,%rax,1)
0x00007fff8724bd60 <__NSAutoreleaseNoPool+48>: int3
0x00007fff8724bd61 <__NSAutoreleaseNoPool+49>: callq 0x7fff872609c2 <dyld_stub_getpid>
0x00007fff8724bd66 <__NSAutoreleaseNoPool+54>: mov %eax,%edi
0x00007fff8724bd68 <__NSAutoreleaseNoPool+56>: mov $0x9,%esi
0x00007fff8724bd6d <__NSAutoreleaseNoPool+61>: leaveq
0x00007fff8724bd6e <__NSAutoreleaseNoPool+62>: jmpq 0x7fff87260a16 <dyld_stub_kill>
0x00007fff8724bd73 <__NSAutoreleaseNoPool+67>: nopw 0x0(%rax,%rax,1)
0x00007fff8724bd79 <__NSAutoreleaseNoPool+73>: nopl 0x0(%rax)
End of assembler dump.
For a practical example, see demo_NSAutoreleaseNoPool.mm.

Can't access /proc/interrupts after free_irq

I am writing a kernel v4l2 driver for an ov7670 CMOS element attached to GPIO pins on a Raspberry Pi. I set up three IRQ lines (Pixel, Href, and Vsync)
Here is how I am requesting them:
ret = request_irq( PX_IRQ,
ov7670rpi_pixel_interrupt,
irq_flags,
"ov7670rpi_px",
ov7670rpi_pixel_interrupt);
ret = request_irq( HREF_IRQ,
ov7670rpi_href_interrupt,
irq_flags,
"ov7670rpi_href",
ov7670rpi_href_interrupt);
ret = request_irq( VSYNC_IRQ,
ov7670rpi_vsync_interrupt,
irq_flags,
"ov7670rpi_vsync",
ov7670rpi_vsync_interrupt);
Now that goes fine:
#cat /proc/interrupts
CPU0
3: 4168 ARMCTRL BCM2708 Timer Tick
9: 0 ARMCTRL ov7670rpipx
10: 0 ARMCTRL ov7670rpihref
11: 0 ARMCTRL ov7670rpivsync
32: 68523 ARMCTRL dwc_otg, dwc_otg_pcd, dwc_otg_hcd:usb1
52: 0 ARMCTRL BCM2708 GPIO catchall handler
65: 543 ARMCTRL ARM Mailbox IRQ
66: 2 ARMCTRL VCHIQ doorbell
75: 1 ARMCTRL
77: 3439 ARMCTRL bcm2708_sdhci (dma)
79: 0 ARMCTRL bcm2708_i2c.0, bcm2708_i2c.1
80: 0 ARMCTRL bcm2708_spi.0
83: 21 ARMCTRL uart-pl011
84: 7436 ARMCTRL mmc0
FIQ: usb_fiq
Err: 0
Looks good.
This is how I disable the IRQs:
/* Disable Interrupts */
free_irq(PX_IRQ, ov7670rpi_pixel_interrupt);
free_irq(HREF_IRQ, ov7670rpi_href_interrupt);
free_irq(VSYNC_IRQ, ov7670rpi_vsync_interrupt);
I have also tried:
/* Disable Interrupts */
free_irq(PX_IRQ, NULL);
free_irq(HREF_IRQ, NULL);
free_irq(VSYNC_IRQ, NULL);
Both ways make it so that once the module is unloaded, I can not access /proc/interrupts. When I try to cat /proc/interrupts, they system locks.
The correct way to do this is to use gpio_request() with your GPIO numbers.
#define PX_GPIO 9
#define HREF_GPIO 10
#define VSYNC_GPIO 11
gpio_request(PX_GPIO, "v4l_rpi_px");
gpio_input(PX_GPIO);
gpio_request(HREF_GPIO, "v4l_rpi_href");
gpio_input(HREF_GPIO);
gpio_request(VSYNC_GPIO, "v4l_rpi_vsync");
gpio_input(VSYNC_GPIO);
/* Now, gpio_to_irq() can be used. */
ret = request_irq(gpio_to_irq(PX_GPIO),
ov7670rpi_pixel_interrupt,
irq_flags,
"ov7670rpi_px",
ov7670rpi_pixel_interrupt);
/* etc. */
bcm2708_gpio.c provides a GPIO interrupt controller as detailed in this question. See also the GPIO documentation.
Your primary interrupt controller ARMCTRL has interrupt 52 as BCM2708 GPIO catchall handler. This IRQ is chained and supports IRQ on each gpio line. The controller for the GPIO's will be listed differently in /proc/interrupts as GPIO.

Why are concrete function implementations in traits compiled to bridge methods in Scala 2.9.x but not in 2.8.x?

In Scala versions prior to 2.9.0, concrete function implementations in traits were compiled as normal methods. From 2.9.x onward, they are compiled as bridge methods. I'm trying to find the reasoning behind this change, because it negatively affects users of many popular Java frameworks like Spring and Jersey.
Consider the following Scala code:
trait Speaks {
def speak() = {
println("woof")
}
}
class Dog extends Speaks {
def wag() = {
println("wag wag")
}
}
When the Dog class is compiled with scalac version 2.8.1 and decompiled with javap, the result for the "speak" and "wag" functions look like this:
public void speak();
flags: ACC_PUBLIC
Code:
stack=1, locals=1, args_size=1
0: aload_0
1: invokestatic #11 // Method Speaks$class.speak:(LSpeaks;)V
4: return
LineNumberTable:
line 7: 0
public void wag();
flags: ACC_PUBLIC
Code:
stack=2, locals=1, args_size=1
0: getstatic #18 // Field scala/Predef$.MODULE$:Lscala/Predef$;
3: ldc #20 // String wag wag
5: invokevirtual #24 // Method scala/Predef$.println:(Ljava/lang/Object;)V
8: return
LineNumberTable:
line 9: 0
When Dog is compiled with scalac version 2.9.1 and again decompiled, the same two functions look like:
public void speak();
flags: ACC_PUBLIC, ACC_BRIDGE
Code:
stack=1, locals=1, args_size=1
0: aload_0
1: invokestatic #11 // Method Speaks$class.speak:(LSpeaks;)V
4: return
LineNumberTable:
line 7: 0
public void wag();
flags: ACC_PUBLIC
Code:
stack=2, locals=1, args_size=1
0: getstatic #18 // Field scala/Predef$.MODULE$:Lscala/Predef$;
3: ldc #20 // String wag wag
5: invokevirtual #24 // Method scala/Predef$.println:(Ljava/lang/Object;)V
8: return
LineNumberTable:
line 9: 0
The problematic part is the addition of the ACC_BRIDGE flag to the speak() function. Frameworks like Jersey and Spring intentionally do not recognize bridge methods in many cases as workarounds for other issues.
So can anyone explain or point to a good explanation of why this change was made in Scala 2.9.x?
As a followup, is there a way to disable this behavior through a function annotation, compiler flag, etc?
OK it sounds like there is no explanation as to why because it wasn't an intentional change. See this thread:
http://groups.google.com/group/scala-language/browse_thread/thread/67f8884081d46912
And the solution is to use the newest snapshot, or, if the future has arrived as you read this, Scala 2.10

Resources