Three.js drag a model on x and z axis. React three fiber - three.js

I am trying to make models draggable in three.js. I want my model to follow my mouse when I move it. This is what I am trying to accomplish. I am using react-three-fiber and #use-gesture/react
What I am trying to accomplish
Here is how my program looks
My program
The difference is quite noticeable. On the good example, the model follows the mouse wherever it goes. On my program, that is not the case.
Here is my code for the cube
const BasicOrangeBox = ({setControlsDisabled, startPosition} : BasicOrangeBoxType) => {
const { camera } = useThree();
const [boxPosition, setBoxPosition] = useState(startPosition)
const bind = useGesture({
onDrag: ({movement: [x, y]}) => {
setControlsDisabled(true);
setBoxPosition( (prev) => {
const newObj = {...prev};
newObj.x = newObj.x0 + (x/100);
newObj.z = newObj.z0 + (y/100);
return newObj;
} )
},
onDragEnd: () => {
setControlsDisabled(false);
setBoxPosition( (prev) => {
const newObj = {...prev};
newObj.x0 = newObj.x;
newObj.z0 = newObj.z;
return newObj;
} )
}
})
return (
<mesh
{...bind() }
position={[boxPosition.x, boxPosition.y, boxPosition.z]}
>
<boxGeometry />
<meshBasicMaterial color={"orange"} />
</mesh>
)
}

Here is how I made a mesh cube draggable only on x and z axis like in a video.
Needed packages:
three
react-three/fiber
use-gesture/react
First, I created a plane that spanned across my whole viewport and assigned it to a useRef
<mesh rotation={[MathUtils.degToRad(90), 0, 0]} ref={planeRef} position={[0, -0.01, 0]}>
<planeGeometry args={[innerWidth, innerHeight]} />
<meshBasicMaterial color={0xfffffff} side={DoubleSide} />
</mesh>
Then I added that ref to a useContext so I can use it in different components.
Next, I imported raycaster from useThree hook and planeRef from aforementioned useContext.
Then I used useGesture onDrag and onDragEnd to enable and disable my OrbitControls
Inside the onDrag, I used raycaster's intersectsObject method and added an array of only one element, my plane, as a parameter. This gave me x, y, z coordinates where my mouse intersects with the plane. (Y is always 0)
Then I updated my box position.
Here is the full code snippet
const BasicOrangeBox = ({setControlsDisabled, startPosition} : BasicRedBoxType) => {
const { raycaster } = useThree();
const [boxPosition, setBoxPosition] = useState(startPosition);
const planeRef = useContext(PlaneContext);
const bind = useGesture({
onDrag: () => {
const intersects = raycaster.intersectObjects([planeRef]);
if (intersects.length > 0){
const intersection = intersects[0];
console.log(intersection.point.x);
setBoxPosition({
x: intersection.point.x,
y: intersection.point.y,
z: intersection.point.z,
})
}
setControlsDisabled(true);
},
onDragEnd: () => {
setControlsDisabled(false);
}
})
return ( //#ts-ignore Ignores type error on next line
<mesh
{...bind() }
position={[boxPosition.x, boxPosition.y, boxPosition.z]}
>
<boxGeometry />
<meshBasicMaterial color={"orange"} />
</mesh>
)
}

Related

Multiple scenes in VR mode and different cameras

I'm trying to implement this fiddle in an XR environment.
In the fiddle, the second scene is fixed to the screen, but not with the oculus/player camera...
Does anyone have a solution to making a scene or an object always on the top right corner of an oculus? I suppose there is a mistake with the size and camera but cannot find what...
I'm not sure, I broke my brain, it's my first step in XR...
Portal Second Scene Code :
function Viewcube() {
const { gl, scene, camera, size } = useThree()
const virtualScene = useMemo(() => new Scene(), [])
const virtualCam = useRef()
const ref = useRef()
const [hover, set] = useState(null)
const matrix = new Matrix4()
useFrame(() => {
matrix.copy(camera.matrix).invert()
ref.current.quaternion.setFromRotationMatrix(matrix)
gl.autoClear = true
gl.render(scene, camera)
gl.autoClear = false
gl.clearDepth()
gl.render(virtualScene, virtualCam.current)
}, 1)
return createPortal(
<>
<OrthographicCamera ref={virtualCam} makeDefault={false} position={[0, 0, 100]} />
<mesh
ref={ref}
raycast={useCamera(virtualCam)}
position={[size.width / 2 - 80, size.height / 2 - 80, 0]}
onPointerOut={(e) => set(null)}
onPointerMove={(e) => set(Math.floor(e.faceIndex / 2))}>
{[...Array(6)].map((_, index) => (
<meshLambertMaterial attachArray="material" key={index} color={hover === index ? 'hotpink' : 'white'} />
))}
<boxGeometry args={[60, 60, 60]} />
</mesh>
<ambientLight intensity={0.5} />
<pointLight position={[10, 10, 10]} intensity={0.5} />
</>,
virtualScene
)
}
To summarize, I would like to display something fixed in the top-right corner of the oculus view, even when moving head, but I am stuck...

How Redux-Saga connect with Babylon-React hook

I've been doing a project to have a redux-saga react pattern to store and display the babylon scene logic, what I thoungt was distributing babylon stuff inside a single js file then export to a react fragment.
My qestion is how can we sent the data generate in babylon js by users, outside of the babylon js file (I have tried things like useState but it seemed that only work on react fragment but my babylon js is only handle for game logic.) I think if I can figure out this, I will be able to do futhur step like conncet with redux-saga.
My purpose is first of all bring the params like the position x,y,z outside createScene.js to be utilized by redux-saga, and if the user refresh the page, the scene he created won't dispear.
React newbie here seeking for suggestion, thanks in advance!
React-babylon hook below
import SceneComponent from 'babylonjs-hook'
import styled from 'styled-components'
import 'App.css'
import { onRender, onSceneReady } from '../hooks/babylonjs/createScene'
const ThreeDEditPageMain = styled.div``
const ThreeDEditPage = () => (
<ThreeDEditPageMain>
<SceneComponent antialias onSceneReady={onSceneReady} onRender={onRender} id="my-canvas" />
</ThreeDEditPageMain>
)
export default ThreeDEditPage
createScene.js below
import {
ActionManager,
ArcRotateCamera,
Color3,
ExecuteCodeAction,
HemisphericLight,
Mesh,
MeshBuilder,
StandardMaterial,
Vector3,
VertexBuffer,
} from '#babylonjs/core'
export const onSceneReady = scene => {
// This creates and positions a free camera (non-mesh)
const camera = new ArcRotateCamera('camera1', 0.4, 0.4, 50, new Vector3(0, 5, -10), scene)
// This targets the camera to scene origin
camera.setTarget(Vector3.Zero())
const canvas = scene.getEngine().getRenderingCanvas()
// This attaches the camera to the canvas
camera.attachControl(canvas, true)
camera.wheelPrecision = 50
// This creates a light, aiming 0,1,0 - to the sky (non-mesh)
const light = new HemisphericLight('light', new Vector3(0, 1, 0), scene)
// Default intensity is 1. Let's dim the light a small amount
light.intensity = 0.7
// Our built-in 'ground' shape.
const ground = MeshBuilder.CreateGround(
'ground',
{ width: 100, height: 100, subdivisions: 100 },
scene,
)
ground.updateFacetData()
// console.log(ground.facetNb)
// Our built-in 'box' shape.
const size = 4
const box = MeshBuilder.CreateBox('box', { size }, scene)
// Move the box upward 1/2 its height
// box.position.y = 1
box.position = new Vector3(size / 2, size / 2, size / 2)
box.bakeCurrentTransformIntoVertices()
box.isPickable = false
const positions = ground.getVerticesData(VertexBuffer.PositionKind)
// console.log(positions)
const snappedPosition = new Vector3()
box.position = snappedPosition
scene.onPointerMove = e => {
const pickingInfo = scene.pick(scene.pointerX, scene.pointerY)
if (pickingInfo.hit && pickingInfo.pickedMesh.name === 'ground') {
snappedPosition.x = Math.round(pickingInfo.pickedPoint.x)
snappedPosition.y = Math.round(pickingInfo.pickedPoint.y)
snappedPosition.z = Math.round(pickingInfo.pickedPoint.z)
}
}
// click action for player
ground.actionManager = new ActionManager(scene)
ground.actionManager.registerAction(
new ExecuteCodeAction(ActionManager.OnPickUpTrigger, () => {
// player clicked
console.log(
`gen a new box at x:${snappedPosition.x}, y:${snappedPosition.y}, z:${snappedPosition.z}`,
)
const genBox = Mesh.CreateBox('box', 4, scene)
genBox.position = new Vector3(snappedPosition.x, snappedPosition.y + 2, snappedPosition.z)
const mat = new StandardMaterial('mat', scene)
mat.diffuseColor = new Color3(Math.random(), Math.random(), Math.random()) // color stuff
genBox.material = mat
}),
)
}
export function onRender(sence) {
}

Applying two different fragment shaders to two different materials (of the same type) using onBeforeCompile?

I've imported a GLTF file with two different meshes. My goal is to give each mesh a material with a unique custom fragment shader using onBeforeCompile. Each mesh has the same type of material (MeshNormalMaterial).
When I try to apply one fragment shader to one material and the other fragment shader to the other material, both materials wind up with the same fragment shader. The fragment shader each material has depends on which material I setup first.
Here's a few pictures showing what I'm talking about:
Below is all the relevant code.
Main code: This is the general structure of my code. I've enclosed the important part between "PRIMARY AREA OF INTEREST" comments. For simplicity, I've replaced my shader code with "..." or a comment describing what it does. They do work as shown in the pictures above.
// Three.JS Canvas
const threeDisplay = document.getElementById("threeDisplay");
// Globals
var displayDimensions = getElemDimensions(threeDisplay); // Uniform
var currentTime = 0; // Uniform
var helix = null; // Mesh
var innerHelix = null; // Mesh
var horseshoe = null; // Mesh
// Set the scene and camera up
const scene = new THREE.Scene();
const camera = initCamera();
// Setup a directional light
const light = new THREE.DirectionalLight( 0xffffff, 1.0 );
light.position.set(-0.2, 1, -0.6);
scene.add(light);
// Setup WebGL renderer
const renderer = initRenderer();
threeDisplay.appendChild( renderer.domElement );
// Load the gltf model
new GLTFLoader().load( "./spiral_pillar_hq_horseshoe.glb", function (object) {
const helixFragmentShaderReplacements = [
{
from: ' ... ',
to: ' // rainbow '
}
];
const horseshoeFragmentShaderReplacements = [
{
from: ' ... ',
to: ' // white '
}
];
//////////////////////////////////////
// PRIMARY AREA OF INTEREST - START //
//////////////////////////////////////
// Turn the horseshoe into a shader.
horseshoe = object.scene.children[1];
var horseshoeGeometry = horseshoe.geometry;
var horseshoeMaterial = shaderMeshMaterial(new THREE.MeshNormalMaterial(), horseshoeGeometry, horseshoeFragmentShaderReplacements);
var horseshoeMesh = new THREE.Mesh(horseshoeGeometry, horseshoeMaterial);
horseshoe = horseshoeMesh;
horseshoe.rotation.z = deg2rad(180); // Re-orient the horseshoe to the correct position and rotation.
horseshoe.position.y = 13;
scene.add(horseshoe);
// Turn the inner helix into a colorful, wiggly shader.
helix = object.scene.children[0];
var helixGeometry = helix.geometry;
var helixMaterial = shaderMeshMaterial(new THREE.MeshNormalMaterial(), helixGeometry, helixFragmentShaderReplacements);
var helixMesh = new THREE.Mesh(helixGeometry, helixMaterial);
helix = helixMesh;
scene.add(innerHelix);
animate();
////////////////////////////////////
// PRIMARY AREA OF INTEREST - END //
////////////////////////////////////
}, undefined, function (error) {
console.error(error);
});
Below are functions which are relevant.
shaderMeshMaterial: Constructs a new material based on the supplied materialType that supports editing the default shader. If it's not initProcessing, then the problem may stem from this function.
// Globals used: displayDimensions
function shaderMeshMaterial(materialType, geometry, fragmentShaderReplacements) {
var material = materialType;
material.onBeforeCompile = function ( shader ) {
// Uniforms
shader.uniforms.time = { value: 0 };
shader.uniforms.resolution = { value: new THREE.Vector2(displayDimensions.width, displayDimensions.height) };
shader.uniforms.bboxMin = { value: geometry.boundingBox.min };
shader.uniforms.bboxMax = { value: geometry.boundingBox.max };
fragmentShaderReplacements.forEach((rep) => {
shader.fragmentShader = shader.fragmentShader.replace(rep.from, rep.to);
});
console.log(shader);
material.userData.shader = shader;
}
return material;
}
initRenderer: Sets up the renderer. Just showing you guys the renderer setup I have in case that's important.
// Globals used: displayDimensions
function initRenderer() {
var renderer = new THREE.WebGLRenderer({
alpha: true,
antialias: true,
precision: "mediump"
});
renderer.setClearColor( 0x000000, 0);
renderer.setPixelRatio( window.devicePixelRatio );
renderer.setSize( displayDimensions.width, displayDimensions.height );
renderer.shadowMap.enabled = true;
renderer.outputEncoding = THREE.sRGBEncoding;
renderer.toneMapping = THREE.ACESFilmicToneMapping;
renderer.toneMappingExposure = 1.25;
return renderer;
}
animate: Handles the animation frames.
// Globals used: renderer, currentTime, postprocessing
function animate (timestamp = 0) {
requestAnimationFrame(animate);
resizeRendererToDisplaySize(renderer);
currentTime = timestamp/1000; // Current time in seconds.
scene.traverse( function ( child ) {
if ( child.isMesh ) {
const shader = child.material.userData.shader;
if ( shader ) {
shader.uniforms.time.value = currentTime;
}
}
} );
renderer.render( scene, camera );
postprocessing.composer.render( 0.1 );
};
One last thing to note is that when I inspected the console log of shader from the shaderMeshMaterial function, I can see that the fragment shaders are indeed different as they should be for each material. Also not sure why there are 4 console logs when there should only be 2.
Sorry for all the code, but I did condense it to where all irrelevant code was stripped out. I'm fairly new to Three.JS, so any possible explanations as to why this is happening are much appreciated!
EDIT: Removed vertex shader parameter from shaderMeshMaterial function to keep this question focused on just the fragment shaders. Though this problem does apply to both the vertex and fragment shaders, I figure if you fix one then you'll fix the other.
EDIT 2: Added language identifiers to code snippets. Also I removed the postprocessing function and the problem still persists, so I know the problem isn't caused by that. I've updated the code above to reflect this change. As a happy side effect of removing the postprocessing function, the console.log of the shader variable from shaderMeshMaterial new appears twice in the log (as it should).
EDIT 3: (Implementing WestLangley's suggestion) I tweaked the shaderMeshMaterial function by adding the customProgramCacheKey function. I had to condense the four parameters of shaderMeshMaterial into one for the sake of the customProgramCacheKey function. I believe I implemented the function correctly, but I'm still getting the same result as before where both materials display the same fragment shader.
New "PRIMARY AREA OF INTEREST" code:
horseshoe = object.scene.children[1];
var horseshoeGeometry = horseshoe.geometry;
var meshData = {
materialType: new THREE.MeshNormalMaterial(),
geometry: horseshoeGeometry,
fragmentShaderReplacements: horseshoeFragmentShaderReplacements
}
var horseshoeMaterial = shaderMeshMaterial(meshData);
var horseshoeMesh = new THREE.Mesh(horseshoeGeometry, horseshoeMaterial);
horseshoe = horseshoeMesh;
horseshoe.rotation.z = deg2rad(180); // Re-orient the horseshoe to the correct position and rotation.
horseshoe.position.y = 13;
scene.add(horseshoe);
// Turn the inner helix into a colorful, wiggly shader.
helix = object.scene.children[0];
var helixGeometry = helix.geometry;
var meshData2 = {
materialType: new THREE.MeshNormalMaterial(),
geometry: helixGeometry,
fragmentShaderReplacements: helixFragmentShaderReplacements
}
var helixMaterial = shaderMeshMaterial(meshData2);
var helixMesh = new THREE.Mesh(helixGeometry, helixMaterial);
helix = helixMesh;
scene.add(innerHelix);
animate();
New shaderMeshMaterial code:
// Globals used: displayDimensions
function shaderMeshMaterial(meshData) {
var material = meshData.materialType;
material.onBeforeCompile = function ( shader ) {
// Uniforms
shader.uniforms.time = { value: 0 };
shader.uniforms.resolution = { value: new THREE.Vector2(displayDimensions.width, displayDimensions.height) };
shader.uniforms.bboxMin = { value: meshData.geometry.boundingBox.min };
shader.uniforms.bboxMax = { value: meshData.geometry.boundingBox.max };
meshData.fragmentShaderReplacements.forEach((rep) => {
shader.fragmentShader = shader.fragmentShader.replace(rep.from, rep.to);
});
material.customProgramCacheKey = function () {
return meshData;
};
console.log(shader);
material.userData.shader = shader;
}
return material;
}
WestLangley suggestion worked for me!
material.onBeforeCompile = ...
// Make sure WebGLRenderer doesnt reuse a single program
material.customProgramCacheKey = function () {
return UNIQUE_PER_MATERIAL_ID;
};
I believe your mistake is returning meshData from customProgramCacheKey.
I think customProgramCacheKey need concrete identifier like a number or string.
It would be nice to understand what exactly happening and why do we need to specify customProgramCacheKey.
EDIT: I discover that default value for customProgramCacheKey calculated as follow in Threejs source.
customProgramCacheKey() {
return this.onBeforeCompile.toString();
}
Perhaps this is explains this default caching behavior because calling toString on function returns that function body literally as string.
For example consider function const myFunc = () => { return 1 }. Calling myFunc.toString() returns "() => { return 1 }"
So if your calling onBeforeCompile in a for loop you function body as string never change.

I want change scrollview rolling speed in react native

Now I use interval make it come true, but it is very incoherence.
If I can just change the method (scroll) speed, it well be nice.
this.interval = setInterval(()=>{
if(!_scroll){
_this.interval && clearInterval(_this.interval);
}
if(totalWide+ScreenWidth >= width ){
_scroll.scrollWithoutAnimationTo();
totalWide=0;
i=0;
}else{
_scroll.scrollTo({x:eachWide*i,animate:true});
totalWide = totalWide + eachWide;
i= i+1;
}
},250)
use decelerationRate property of ScrollView
<ScrollView decelerationRate={0.5}>
</ScrollView>
I got this working by having setInterval call a function(in which you define the logic or the pace at which the scroll should move).
this.interval= setInterval(this.scrollwithSpeed, 100); // Set the function to this timer
scrollwithSpeed() {
position = this.state.currentPosition + x; // x decides the speed and
currentPosition is set to 0 initially.
this.scrollObject.scrollTo(
{ y: position, animated: true }
);
this.setState({ currentPosition: position });
}
Make sure you call clearInterval(this.interval) after it is done.
I would suggest to attach to js requestAnimationFrame (from how far I know it is supported in React Native).
Bellow example will scroll linearly from top to bottom. If You need to scoll to different offset just change distance variable.
startingPoint variable is redundant in scrolling from top to bottom but will stay in example.
scroll() {
if (this.scrollAnimationFrame) {
cancelAnimationFrame(this.scrollAnimationFrame);
}
this.listRef.scrollToOffset({offset: 0, animated: false}); // remove if You don't start scroll from top
const duration = this.scrollTime,
startingPoint = 0, // change if You don't start scroll from top
distance = Scrolling.LINE_HEIGHT * Scrolling.ITEMS_COUNT;
let startTimestamp, progress;
const frameCallback = (timestamp) => {
if (!startTimestamp) {
startTimestamp = timestamp;
}
progress = timestamp - startTimestamp;
this.listRef.scrollToOffset({
offset: distance * (progress / duration) + startingPoint,
animated: false,
});
if (progress < duration) {
this.scrollAnimationFrame = requestAnimationFrame(frameCallback);
}
};
this.scrollAnimationFrame = requestAnimationFrame(frameCallback);
}
You can use reanimated to make it work.
const offsetY = useSharedValue(0);
const animatedProps = useAnimatedProps<FlatListProps<unknown>>(() => {
return {
contentOffset: {
x: 0,
y: offsetY.value,
},
};
});
const handleScroll = () => {
offsetY.value = withTiming(targetIndex * CARD_HEIGHT, {
duration: YOUR_DURATION_HERE,
});
}
return <Animated.FlatList animatedProps={animatedProps} ... />

Three.js make the text created with THREE.ShapeGeometry face the camera

I created a shapegeometry with the text. How can I keep the text face the camera on move the camera?
...
this.textGeometry = new THREE.ShapeGeometry(THREE.FontUtils.generateShapes(value, parameters));
this.textValue = new THREE.Mesh(this.textGeometry, new THREE.MeshBasicMaterial({ color: color, side: THREE.DoubleSide }));
this.textValue.matrixAutoUpdate = true;
this.add(this.textValue)
...
I think my problem is that I modified the parent quaternion 3D object:
this.quaternion.setFromAxisAngle (axis, radians);
then the only operation:
textValue.quaternion.copy (camera.quaternion);
is not sufficient
how can I fix the rotation considering the state of the quaternion?
If you don't care about calling the base updateMatrix function,
this can be a solution
yourShapeGeometry.prototype.updateMatrix = function(){
// THREE.Object3D.prototype.updateMatrix.call(this);
fixOrientation(this.textValue);
}
function fixOrientation(mesh){
mesh.setRotationFromQuaternion(camera.quaternion);
mesh.updateMatrix();
}
or simply edit the updateMatrix of your text mesh like
textMesh.updateMatrixWorld = updateSpriteWorld;
function updateSpriteWorld(){
if ( this.matrixWorldNeedsUpdate === true || force === true ) {
this.setRotationFromQuaternion(camera.quaternion);
this.updateMatrix();
this.matrixWorld.copy( this.matrix );
this.matrixWorldNeedsUpdate = false;
force = true;
}
// update children
for ( var i = 0, l = this.children.length; i < l; i ++ ) {
this.children[ i ].updateSpriteWorld( force );
}
}
I think this should do the trick:
this.textValue.lookAt( camera.position );

Resources