I have a dataset of categorical and continuos features and a lot of them have missing elements. I was wondering if i can use the respective imputer to fill out continuos as well as categorical data.
And if it cant be done, what would be the best way to proceed? Would it be best to separate the data into continuos and discrete features and then use, for example, IterativeImputer for the first set and KNN for the second one and then merge them?
Any help would be appreciated.
The data consists of 65 features:
x_train
age sex painloc painexer relrest cp trestbps htn chol smoke ... om1 om2 rcaprox rcadist lvx1 lvx2 lvx3 lvx4 lvf cathef
288 -1.109572 1.0 0.0 0.0 0.0 1.0 -0.655059 0.0 0.818661 NaN ... NaN NaN NaN NaN 1.0 1.0 1.0 1.0 2.0 0.568676
283 -0.180525 1.0 1.0 0.0 0.0 2.0 1.447445 0.0 -0.040919 NaN ... NaN NaN NaN NaN 1.0 1.0 1.0 1.0 1.0 NaN
230 -0.077297 1.0 1.0 1.0 0.0 3.0 0.659006 1.0 2.872604 NaN ... 2.0 NaN 2.0 NaN 1.0 1.0 1.0 1.0 1.0 NaN
380 -0.799890 0.0 1.0 1.0 1.0 4.0 -0.129433 0.0 0.339106 NaN ... NaN NaN NaN NaN 1.0 1.0 1.0 1.0 1.0 NaN
147 0.129157 1.0 1.0 1.0 1.0 4.0 NaN 0.0 0.031467 0.0 ... 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 -0.822164
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
847 -0.180525 0.0 NaN NaN NaN 3.0 0.185942 1.0 -0.040919 NaN ... 1.0 NaN 1.0 1.0 1.0 1.0 1.0 1.0 1.0 NaN
301 -0.283752 1.0 1.0 1.0 1.0 4.0 -0.129433 0.0 -0.194738 NaN ... NaN NaN NaN NaN 1.0 1.0 1.0 1.0 1.0 NaN
693 0.645295 1.0 NaN NaN NaN 4.0 -0.392246 1.0 0.520070 NaN ... 1.0 NaN 2.0 1.0 1.0 1.0 1.0 1.0 1.0 NaN
115 1.058204 1.0 1.0 1.0 1.0 4.0 NaN 0.0 0.954384 0.0 ... 1.0 1.0 2.0 1.0 1.0 1.0 1.0 1.0 1.0 -0.811925
155 1.574341 1.0 1.0 1.0 1.0 4.0 NaN 1.0 NaN 0.0 ... 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 1.0 NaN
I have standarized the continuos variables.
There are many categorical features like 'painloc' a 'painexer' that have missing values, and there are also continous ones like 'age' (i decided to treat it as continuous) and 'chol' that also have missing elements.
I tried using IterativeImputer:
x_mice=x_train
mice_impute = IterativeImputer(sample_posterior=True)
x_mice=pd.DataFrame(mice_impute.fit_transform(x_mice))
x_mice.columns=labels
x_mice
age sex painloc painexer relrest cp trestbps htn chol smoke ... om1 om2 rcaprox rcadist lvx1 lvx2 lvx3 lvx4 lvf cathef
0 1.049449 1.0 1.000000 1.000000 1.000000 4.0 0.444874 0.000000 0.540723 0.000000 ... 1.000000 1.000000 2.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 -0.891887
1 0.505617 1.0 1.000000 1.000000 0.000000 2.0 -0.266785 0.000000 -1.752150 0.000000 ... 1.000000 1.000000 2.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 -0.888760
2 0.831916 1.0 1.000000 0.000000 0.000000 4.0 -1.080109 0.764037 -1.752150 1.450166 ... 1.000000 1.000000 1.000000 1.000000 1.025761 0.879404 -0.400332 3.193691 3.267492 1.118696
3 -0.582047 1.0 1.000000 0.000000 0.000000 2.0 -1.588436 0.000000 -0.249794 0.000000 ... 1.383778 1.048614 -0.147575 1.942328 1.000000 1.000000 1.000000 1.000000 1.000000 0.802084
4 -1.452178 1.0 1.000000 0.000000 0.000000 3.0 0.444874 1.000000 5.232542 1.000000 ... 1.235595 1.249215 2.269437 1.155985 1.000000 1.000000 1.000000 1.000000 1.000000 -1.935223
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
624 0.179318 1.0 1.000000 1.000000 1.000000 4.0 -0.571781 0.000000 0.628910 -0.060307 ... 0.928614 0.830982 1.080936 1.185430 1.000000 1.000000 1.000000 1.000000 1.000000 -1.032691
625 1.702047 1.0 1.000000 0.000000 1.000000 3.0 0.444874 0.000000 -1.752150 0.000000 ... 1.000000 1.000000 2.000000 1.000000 1.000000 1.000000 1.000000 1.000000 2.000000 -0.895014
626 -0.364514 1.0 0.694690 1.738101 0.396025 4.0 0.953201 1.000000 0.390804 1.287500 ... 1.000000 0.739708 2.000000 1.000000 1.000000 1.000000 1.000000 1.000000 2.000000 -0.523902
627 0.723149 1.0 0.762459 0.038032 0.315826 4.0 0.444874 1.000000 0.831741 0.750375 ... 1.000000 0.912221 2.000000 1.000000 1.000000 1.000000 1.000000 1.000000 2.000000 0.730936
628 0.940682 1.0 1.000000 1.000000 1.000000 4.0 -0.000217 0.000000 -0.252964 0.000000 ... 1.000000 1.000000 2.000000 1.000000 1.000000 1.000000 1.000000 1.000000 1.000000 -0.888134
It works fine for continuos features but not for categorical as it can fill in a decimal number and thats obviously not right.
Related
I have exported a simple box mesh from blender as wrl. I have "smart uv unwrap"ed the mesh also. I am using this mesh only for it's geometry and am applying material through threejs. I am then binding a texture with the imported mesh, and i am also setting the needsUpdate property of material (MeshPhongMaterial) to true.
The result is as follows :
The applied texture :
The code excerpts are as follows :
function generateCustomTHREEMeshFromWRL(name) {
return vrmlLoader.parse(getResource(name).content).children[0]
//the getResource(name).content returns the entire wrl file content
}
mesh = generateCustomTHREEMeshFromWRL('unwrapped.wrl')
mesh.material.map = textureLoader.load('lavatile.jpg')
mesh.material.needsUpdate = true;
scene.add(mesh)
The WRL file is as follows :
#VRML V2.0 utf8
#modeled using blender3d http://blender.org
# 'Cube'
Shape {
appearance Appearance {
material Material {
}
}
geometry IndexedFaceSet {
coord Coordinate {
point [ -1.000000 -1.000000 1.000000 -1.000000 -1.000000 -1.000000 1.000000 -1.000000 -1.000000 1.000000 -1.000000 1.000000 -1.000000 1.000000 0.999999 -0.999999 1.000000 -1.000001 1.000000 1.000000 -1.000000 1.000000 1.000000 1.000000 ]
}
colorPerVertex FALSE
color Color {
color [ 0.80 0.80 0.80 ]
}
colorIndex [ 0 0 0 0 0 0 0 0 0 0 0 0 ]
coordIndex [ 1 3 0 -1 7 5 4 -1 4 1 0 -1 5 2 1 -1 2 7 3 -1 0 7 4 -1 1 2 3 -1 7 6 5 -1 4 5 1 -1 5 6 2 -1 2 6 7 -1 0 3 7 -1 ]
}
}
This is a glimpse of the actions I am performing in blender
(Note: I've posted this on the Clara.io Forums and while it's only been a day, I've not received any replies. I figured I'd try StackOverflow and update my Clara.io post.)
I'm hoping (guessing) that this is a quick-fix and that I'm missing something basic, but I've searched for quite a bit of time looking for a solution. I'm seeing my final render has the color grey in place of the assigned material, whereas all other non-extruded faces render fine.
I'm learning the Clara.io interface, (along with the basic essentials of 3D modeling really), and have a basic cube with a few extruded faces. In the screenshot below you can see the normals of all faces, including the extruded ones. This is a result of Dr. Google saying "look at your normals, they're probably flipped". It doesn't appear so.
While I put this in the "Rendering" category, I could've also applied it to "ThreeJS" as I'm wondering if it's something I've configured wrong in the ThreeJS scene. I'm using a MeshStandardMaterial along with a normal and roughness map:
var materialTest5 = new THREE.MeshStandardMaterial( {
normalMap: textureLoaderQ.load( "/textures/metal18/metal_plates_18_nm.jpg" ),
roughnessMap: textureLoaderQ.load( "/textures/metal18/metal_plates_18_rough.jpg" ),
map: textureLoaderQ.load( "/textures/metal18/metal_plates_18_dif.jpg" )
} );
var loader5 = new THREE.OBJLoader();
loader5.load( "/models/capsule/test-01.obj", function ( group ) {
for ( var index = 0; index < group.children.length; index++ ) {
var geometry = group.children[ index ].geometry;
var meshTest5 = new THREE.Mesh( geometry, materialTest5 );
scene.add( meshTest5 );
}
} );
They appear like this:
You can see the grey faces. Try as I might, I can't get those faces to render using the same maps as the other faces. I've tried flipping normals, I've tried altering the material in ClaraIO, and a few other settings that were futile.
Within the Clara.io, I can see a similar situation:
Anyone see this before? Have an idea?
Thank you.
Jon
[UPDATE] Here are the OBJ and MTL files I'm using:
OBJ
MTL
MTL
# Blender MTL File: 'None'
# Material Count: 1
newmtl Standard
Ns 96.078431
Ka 0.000000 0.000000 0.000000
Kd 1.000000 1.000000 1.000000
Ks 0.000000 0.000000 0.000000
Ni 1.000000
d 1.000000
illum 2
map_Kd metal_plates_18_dif.jpg
OBJ (Not too many points, so I figured I could add it here too).
# Blender v2.74 (sub 0) OBJ File: ''
# www.blender.org
mtllib test-01.mtl
o Box
v 0.500000 0.500000 0.500000
v 0.500000 0.500000 0.000000
v 0.500000 0.500000 -0.500000
v 0.500000 0.000000 0.500000
v 0.500000 0.000000 0.000000
v 0.500000 0.000000 -0.500000
v 0.500000 -0.500000 0.500000
v 0.500000 -0.500000 0.000000
v 0.500000 -0.500000 -0.500000
v -0.500000 0.500000 -0.500000
v -0.500000 0.500000 0.000000
v -0.615470 0.615470 0.615470
v -0.500000 0.000000 -0.500000
v -0.500000 0.000000 0.000000
v -0.500000 0.000000 0.500000
v -0.500000 -0.500000 -0.500000
v -0.500000 -0.500000 0.000000
v -0.500000 -0.500000 0.500000
v 0.000000 0.500000 -0.500000
v 0.000000 0.500000 0.000000
v 0.000000 0.506330 0.500000
v 0.000000 -0.500000 0.500000
v 0.000000 -0.500000 0.000000
v 0.000000 -0.500000 -0.500000
v 0.000000 0.000000 0.500000
v 0.000000 0.000000 -0.500000
v -0.615470 0.115470 0.115470
v -0.615470 0.615470 0.115470
v -0.615470 0.115470 0.615470
v -0.115470 0.615470 0.615470
v -0.115470 0.615470 0.115470
v -0.115470 0.115470 0.615470
vt 0.000000 1.000000
vt 0.000000 0.500000
vt 0.500000 0.500000
vt 0.500000 1.000000
vt 1.000000 0.500000
vt 1.000000 1.000000
vt 0.000000 0.000000
vt 0.500000 0.000000
vt 1.000000 0.000000
vn 1.000000 0.000000 0.000000
vn -1.000000 0.000000 0.000000
vn 0.000000 1.000000 0.000000
vn 0.000000 -1.000000 0.000000
vn 0.000000 0.000000 1.000000
vn 0.000000 0.000000 -1.000000
vn -0.707100 0.000000 -0.707100
vn -0.707100 -0.707100 0.000000
vn 0.707100 0.707100 -0.000000
vn 0.000000 0.707100 -0.707100
vn 0.000000 -0.707100 0.707100
vn 0.707100 0.000000 0.707100
usemtl Standard
s 1
f 1/1/1 4/2/1 5/3/1 2/4/1
f 2/4/1 5/3/1 6/5/1 3/6/1
f 4/2/1 7/7/1 8/8/1 5/3/1
f 5/3/1 8/8/1 9/9/1 6/5/1
f 10/1/2 13/2/2 14/3/2 11/4/2
f 28/4/2 27/3/2 29/5/2 12/6/2
f 13/2/2 16/7/2 17/8/2 14/3/2
f 14/3/2 17/8/2 18/9/2 15/5/2
f 10/1/3 11/2/3 20/3/3 19/4/3
f 19/4/3 20/3/3 2/5/3 3/6/3
f 28/2/3 12/7/3 30/8/3 31/3/3
f 20/3/3 21/8/3 1/9/3 2/5/3
f 18/1/4 17/2/4 23/3/4 22/4/4
f 22/4/4 23/3/4 8/5/4 7/6/4
f 17/2/4 16/7/4 24/8/4 23/3/4
f 23/3/4 24/8/4 9/9/4 8/5/4
f 12/1/5 29/2/5 32/3/5 30/4/5
f 21/4/5 25/3/5 4/5/5 1/6/5
f 15/2/5 18/7/5 22/8/5 25/3/5
f 25/3/5 22/8/5 7/9/5 4/5/5
f 3/1/6 6/2/6 26/3/6 19/4/6
f 19/4/6 26/3/6 13/5/6 10/6/6
f 6/2/6 9/7/6 24/8/6 26/3/6
f 26/3/6 24/8/6 16/9/6 13/5/6
f 11/7/7 14/7/7 27/7/7 28/7/7
f 14/7/8 15/7/8 29/7/8 27/7/8
f 21/7/9 20/7/9 31/7/9 30/7/9
f 20/7/10 11/7/10 28/7/10 31/7/10
f 15/7/11 25/7/11 32/7/11 29/7/11
f 25/7/12 21/7/12 30/7/12 32/7/12
[UPDATE 2] Problem solved thanks to a comment #manthrax made below.
I rebuilt the UV map using the Clara.io "UV Map" option in the Object Tools section. I also had to set the "Projection" to "Box" so the newly created extruded faces didn't have a skewed/angled diffuse layout. While initially I had only included a "diffuse" image, I added both a "normal" and "specular" map this time around. The image below is a bit bright, and loses the metallicity of my original render above, but that's because I put the specular to white (#ffffff). I'll play around with those settings next. Here's the final result:
I also wanted to note that I had to change the Three JS code a bit, to handle the MTL file that Clara.io generated:
var meshZ = null;
var mtlLoader = new THREE.MTLLoader();
mtlLoader.setPath( "/models/capsule/" );
mtlLoader.load( 'test-01.mtl', function( materials ) {
materials.preload();
var objLoader = new THREE.OBJLoader();
objLoader.setMaterials( materials );
objLoader.setPath( "/models/capsule/" );
objLoader.load( 'test-01.obj', function ( object ) {
meshZ = object;
scene.add( meshZ );
} );
} );
Thanks again, #manthrax.
Onto the next 3D OBJ creation learning curve...
In you loader.load callback.. instead of looping through children, can you try:
group.traverse((e)=>{if(e.material)e.material = materialTest5;})
instead of looping through children?
Also, could you post your .OBJ file so we can take a look?
I have read a lot of questions and articles about this issue, but anyway png images for models are not transparent. Instead it has white background. Like:
three.js: how to apply alpha-map when using OBJMTL loader?
Assigning Alpha channels to OBJ/MTL model meshes
Three.js png texture - alpha renders as white instead as transparent
Maybe someone has ideas how to fix it and what code to add?
renderer = new THREE.WebGLRenderer({ antialias: true, alpha: true });
var loader = new THREE.OBJMTLLoader();
loader.crossOrigin = 'anonymous';
loader.load( 'model.obj', 'model.mtl', function ( object ) {
object.position.set(0, 0, 0);
object.traverse( function ( child ) {
if ( child instanceof THREE.Mesh ) {
child.castShadow = true;
child.receiveShadow = true;
}
if ( child.material instanceof THREE.MeshPhongMaterial ) {
child.material.opacity = 1;
// child.material.alphaTest = 0.5;
// child.material.depthWrite = false;
// child.material.depthTest = false;
// child.material.side = THREE.BackSide;
child.material.transparent = true;
}
});
scene.add(object);
});
model.mtl:
newmtl Cap
Ns 37.6553
Ni 1.5000
d 0.9990
Tr 0.0010
Tf 0.9990 0.9990 0.9990
illum 2
Ka 0.7333 0.7333 0.7333
Kd 1.0000 1.0000 1.0000
Ks 0.0980 0.0980 0.0980
Ke 0.0000 0.0000 0.0000
map_Kd Cap_01.png
map_bump Cap_01_Normal.png
bump Cap_01_Normal.png
newmtl Bark
Ns 37.6553
Ni 1.5000
d 1.0000
Tr 0.0000
Tf 1.0000 1.0000 1.0000
illum 2
Ka 0.7333 0.7333 0.7333
Kd 1.0000 1.0000 1.0000
Ks 0.0980 0.0980 0.0980
Ke 0.0000 0.0000 0.0000
map_Kd BroadleafBark.png
map_bump BroadleafBark_Normal.png
bump BroadleafBark_Normal.png
newmtl Leaves
Ns 37.6553
Ni 1.5000
d 1.0000
Tr 0.0000
Tf 1.0000 1.0000 1.0000
illum 2
Ka 0.7647 0.7647 0.7647
Kd 1.0000 1.0000 1.0000
Ks 0.4118 0.4118 0.4118
Ke 0.0000 0.0000 0.0000
map_Kd BroadleafLeaves.png
map_Ks BroadleafLeaves_Spec.png
map_d BroadleafLeaves_Alpha.png
map_bump BroadleafLeaves_Normal.png
bump BroadleafLeaves_Normal.png
Finally! I found solution. I have converted .obj to .js loader with authors of threejs python script (How to convert object to json file for three.js model loader) and then use JSONLoader():
var loader = new THREE.JSONLoader();
loader.load('model.js', function(geometry, materials) {
var material = new THREE.MeshFaceMaterial(materials);
var object = new THREE.Mesh(geometry, material);
...
I need a basic material in my mesh.
I have this material in my MTL file:
newmtl window
Ns 0.000000
Ka 0.000000 0.000000 0.000000
Kd 0.888930 0.994660 0.000000
Ks 0.000000 0.000000 0.000000
Ni 1.000000
d 1.000000
illum 1
what attribute makes material to be "THREE Basic Material"?
The given MTL file translates to
new THREE.MeshLambertMaterial( { color: new THREE.Color( 0.888930, 0.994660, 0 ) } );
I've successfully loaded a 3d model that I triangulated using Blender. Trouble is its all black and I can't figure out how to get it to show color or textures.
I have an ambient light initialized like this:
function init() {
scene = new THREE.Scene;
camera = new THREE.PerspectiveCamera(
75, (window.innerWidth) / (window.innerHeight),
1, 10000);
camera.position.x = 500;
camera.position.z = 100;
camera.lookAt(new THREE.Vector3(0, 0, 0));
camera.rotateZ(90 * Math.PI / 180);
scene.add(new THREE.AmbientLight(0xffffff));
populate();
renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
renderer.render(scene, camera);
document.body.appendChild(renderer.domElement);
document.addEventListener('keydown', function(event) { keydown(event);
});
}
Here's the .mtl that goes with the .obj
Blender MTL File: 'mq9.blend'
Material Count: 4
newmtl Material__69
Ns 9.803922
Ka 0.000000 0.000000 0.000000
Kd 0.225882 0.225882 0.225882
Ks 0.900000 0.900000 0.900000
Ni 1.000000
d 1.000000
illum 2
newmtl _6___Default
Ns 19.607843
Ka 0.000000 0.000000 0.000000
Kd 0.800000 0.800000 0.800000
Ks 0.900000 0.900000 0.900000
Ni 1.000000
d 1.000000
illum 2
map_Kd missile.jpg
newmtl mq_9
Ns 31.372549
Ka 0.000000 0.000000 0.000000
Kd 0.800000 0.800000 0.800000
Ks 0.990000 0.990000 0.990000
Ni 1.000000
d 1.000000
illum 2
map_Kd predator.jpg
map_Bump predator_Normal.jpg
newmtl mq_90
Ns 31.372549
Ka 0.000000 0.000000 0.000000
Kd 0.800000 0.800000 0.800000
Ks 0.990000 0.990000 0.990000
Ni 1.000000
d 1.000000
illum 2
map_Kd predator.jpg
map_Bump predator_Normal.jpg
The various .jpgs are all found but for some reason they don't seem to be rendering by three.js. I'm wondering if the jpgs need to be modified somehow to reflect the triangulation?
Screenshot: