This content originally appeared on DEV Community and was authored by Devs Daddy
Introduction
An upscaling algorithm is an algorithm for improving the quality of real-time animations and images using simple and efficient shaders. Unlike AMD FSR, which is generally focused on PCs and consoles, we will create a plugin that has been designed specifically for upscaling animations, preserving clarity and contrast while working on all platforms.
To implement the plugin in Three.js based on the idea of upscaling, we need to create a shader that will apply filtering and image enhancement in real time. The plugin will work cross-platform thanks to WebGL.
You can view the full source of this tutorial at GitHub:
https://github.com/DevsDaddy/threejs-upscaler
Step 1: Create a shader for upscaling
We need to create a shader that implements the basic upscaling principles, such as contour enhancement and border smoothing, to increase the clarity of the image.
Here is an example of a simple shader in GLSL:
// Vertex Shader
varying vec2 vUv;
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
// Fragment Shader
uniform sampler2D tDiffuse;
uniform vec2 resolution;
varying vec2 vUv;
void main() {
vec2 texelSize = 1.0 / resolution;
// Get neighbor pixels
vec4 color = texture2D(tDiffuse, vUv);
vec4 colorUp = texture2D(tDiffuse, vUv + vec2(0.0, texelSize.y));
vec4 colorDown = texture2D(tDiffuse, vUv - vec2(0.0, texelSize.y));
vec4 colorLeft = texture2D(tDiffuse, vUv - vec2(texelSize.x, 0.0));
vec4 colorRight = texture2D(tDiffuse, vUv + vec2(texelSize.x, 0.0));
// Work with edges
float edgeStrength = 1.0 - smoothstep(0.1, 0.3, length(color.rgb - colorUp.rgb));
edgeStrength += 1.0 - smoothstep(0.1, 0.3, length(color.rgb - colorDown.rgb));
edgeStrength += 1.0 - smoothstep(0.1, 0.3, length(color.rgb - colorLeft.rgb));
edgeStrength += 1.0 - smoothstep(0.1, 0.3, length(color.rgb - colorRight.rgb));
edgeStrength = clamp(edgeStrength, 0.0, 1.0);
// Apply filtering
vec3 enhancedColor = mix(color.rgb, vec3(1.0) - (1.0 - color.rgb) * edgeStrength, 0.5);
gl_FragColor = vec4(enhancedColor, color.a);
}
This shader enhances object boundaries by increasing the contrast between neighboring pixels, which creates a smoothing effect and enhances details.
Step 2: Create a plugin for Three.js
Now let's integrate this shader into Three.js as a plugin that can be used to upscale the scene in real time.
import * as THREE from 'three';
// Simple Upscaling Plugin
class UpscalerPlugin {
constructor(renderer, scene, camera, options = {}) {
this.renderer = renderer;
this.scene = scene;
this.camera = camera;
// Plugin Settings
this.options = Object.assign({
useEdgeDetection: true, // Edge Detection
scaleFactor: 2.0, // Upscaling Value
}, options);
this.onSceneDraw = null;
this.onRender = null;
this.initRenderTargets();
this.initShaders();
}
initRenderTargets() {
// Create render textures
const renderTargetParams = {
minFilter: THREE.LinearFilter,
magFilter: THREE.LinearFilter,
format: THREE.RGBAFormat,
stencilBuffer: false,
};
const width = this.renderer.domElement.width / this.options.scaleFactor;
const height = this.renderer.domElement.height / this.options.scaleFactor;
this.lowResTarget = new THREE.WebGLRenderTarget(width, height, renderTargetParams);
this.highResTarget = new THREE.WebGLRenderTarget(width * this.options.scaleFactor, height * this.options.scaleFactor, renderTargetParams);
}
// Init Upscaling Shaders
initShaders() {
this.upscalerShader = {
uniforms: {
'tDiffuse': { value: null },
'resolution': { value: new THREE.Vector2(this.renderer.domElement.width, this.renderer.domElement.height) },
},
vertexShader: `
varying vec2 vUv;
void main() {
vUv = uv;
gl_Position = projectionMatrix * modelViewMatrix * vec4(position, 1.0);
}
`,
fragmentShader: `
uniform sampler2D tDiffuse;
uniform vec2 resolution;
varying vec2 vUv;
void main() {
vec2 texelSize = 1.0 / resolution;
// Get Neighbor Pixels
vec4 color = texture2D(tDiffuse, vUv);
vec4 colorUp = texture2D(tDiffuse, vUv + vec2(0.0, texelSize.y));
vec4 colorDown = texture2D(tDiffuse, vUv - vec2(0.0, texelSize.y));
vec4 colorLeft = texture2D(tDiffuse, vUv - vec2(texelSize.x, 0.0));
vec4 colorRight = texture2D(tDiffuse, vUv + vec2(texelSize.x, 0.0));
// Work with edges
float edgeStrength = 1.0 - smoothstep(0.1, 0.3, length(color.rgb - colorUp.rgb));
edgeStrength += 1.0 - smoothstep(0.1, 0.3, length(color.rgb - colorDown.rgb));
edgeStrength += 1.0 - smoothstep(0.1, 0.3, length(color.rgb - colorLeft.rgb));
edgeStrength += 1.0 - smoothstep(0.1, 0.3, length(color.rgb - colorRight.rgb));
edgeStrength = clamp(edgeStrength, 0.0, 1.0);
// Applying edges incresing and filtering
vec3 enhancedColor = mix(color.rgb, vec3(1.0) - (1.0 - color.rgb) * edgeStrength, 0.5);
gl_FragColor = vec4(enhancedColor, color.a);
}
`
};
this.upscalerMaterial = new THREE.ShaderMaterial({
uniforms: this.upscalerShader.uniforms,
vertexShader: this.upscalerShader.vertexShader,
fragmentShader: this.upscalerShader.fragmentShader
});
// Generate Sample Scene
if(!this.onSceneDraw){
this.fsQuad = new THREE.Mesh(new THREE.PlaneGeometry(2, 2), this.upscalerMaterial);
this.sceneRTT = new THREE.Scene();
this.cameraRTT = new THREE.OrthographicCamera(-1, 1, 1, -1, 0, 1);
this.sceneRTT.add(this.fsQuad);
}else{
this.onSceneDraw();
}
}
render() {
// Render scene at low resolution
this.renderer.setRenderTarget(this.lowResTarget);
this.renderer.render(this.scene, this.camera);
// apply upscaling
this.upscalerMaterial.uniforms['tDiffuse'].value = this.lowResTarget.texture;
this.upscalerMaterial.uniforms['resolution'].value.set(this.lowResTarget.width, this.lowResTarget.height);
// Render to window
this.renderer.setRenderTarget(null);
if(!this.onRender)
this.renderer.render(this.sceneRTT, this.cameraRTT);
else
this.onRender();
}
}
export { UpscalerPlugin };
Step 3: Using the plugin in the project
Now let's integrate the plugin into our Three.js project.
import * as THREE from 'three';
import { UpscalerPlugin } from './upscaler.js';
// Create Renderer
const renderer = new THREE.WebGLRenderer();
renderer.setSize(window.innerWidth, window.innerHeight);
document.body.appendChild(renderer.domElement);
// Create Three Scene
const scene = new THREE.Scene();
const camera = new THREE.PerspectiveCamera(75, window.innerWidth / window.innerHeight, 0.1, 1000);
camera.position.z = 5;
// Create Geometry
const geometry = new THREE.BoxGeometry();
const material = new THREE.MeshBasicMaterial({ color: 0x00ff00 });
const cube = new THREE.Mesh(geometry, material);
scene.add(cube);
// Initialize Our Upscaler Plugin
const upscaler = new UpscalerPlugin(renderer, scene, camera, {
scaleFactor: 1.25,
useEdgeDetection: true
});
// Initialize stats monitor
const stats = new Stats();
stats.showPanel(0); // 0: FPS, 1: MS, 2: MB
document.body.appendChild(stats.dom);
// On Window Resizing
function onWindowResize() {
renderer.setSize(window.innerWidth, window.innerHeight);
camera.aspect = window.innerWidth / window.innerHeight;
camera.updateProjectionMatrix();
// Update Upscaler Parameters
upscaler.initRenderTargets();
upscaler.initShaders();
}
window.addEventListener('resize', onWindowResize, false);
function animate() {
stats.begin();
requestAnimationFrame(animate);
// Animate Sample Cube
cube.rotation.x += 0.01;
cube.rotation.y += 0.01;
// Render with upscaling
upscaler.render();
stats.end();
}
animate();
Step 4: Cross-platform support
The plugin uses WebGL, which is supported by most modern browsers and devices, making it cross-platform. It will work correctly on both desktops and mobile devices.
Conclusion
This plugin for Three.js provides real-time scene upscaling using GPU shader processing approach. It improves contrast and border clarity, which is especially useful for animated scenes or low-resolution renders. The plugin integrates easily into existing projects and provides cross-platform performance by utilizing WebGL.
You can view the full source at GitHub:
https://github.com/DevsDaddy/threejs-upscaler
You can also help me out a lot in my plight and support the release of new articles and free for everyone libraries and assets for developers:
My Discord | My Blog | My GitHub
BTC: bc1qef2d34r4xkrm48zknjdjt7c0ea92ay9m2a7q55
ETH: 0x1112a2Ef850711DF4dE9c432376F255f416ef5d0
USDT (TRC20): TRf7SLi6trtNAU6K3pvVY61bzQkhxDcRLC
This content originally appeared on DEV Community and was authored by Devs Daddy
Devs Daddy | Sciencx (2024-09-01T19:05:37+00:00) Writing a plugin for upscaling rendering under ThreeJS. Retrieved from https://www.scien.cx/2024/09/01/writing-a-plugin-for-upscaling-rendering-under-threejs/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.