This content originally appeared on Bits and Pieces - Medium and was authored by Lee Martin
Introducing the Twilight Receiver
Volodymyr Agafonkin is a prolific open-source developer who I’ve had the pleasure of conversing with given my collaborations with his employer Mapbox in the past. He is also a rock musician, baker, photographer, martial artist, sci-fi buff, loving husband, father to two 8-year old twin girls, and a Ukrainian. In February of this year, Russia invaded his homeland and he was forced to evacuate his home of Kyiv with his family as he could not bear the thought of his girls experiencing bombings. Since then they have stayed safe in temporary locations throughout Ukraine by heeding the alert of air sirens and enduring sleepless nights in bomb shelters.
When Third Man Records approached me about building an interactive experience for Jack White’s new album Fear of the Dawn, I immediately thought about Volodymyr, his work, and the incredible people of Ukraine. What does it mean to fear the dawn? For most of us, that probably means fearing the idea of having to wake up early but for the Ukrainian people, waking up each day to new destruction and death has been a nightmare. One of the most important things you can do to fight back against Russia is to not be silent. Continue to protest this war, wherever you are, and demand more support for the Ukrainian people from your leaders.
One of my favorite projects of Volodymyr’s is a little open-source JavaScript library called SunCalc. By providing a time and location, SunCalc will return the position of the sun. I had the thought to build a web app that required fans to wake up at dawn in order to hear unreleased music from Jack’s new album.
Our web app, the Twilight Receiver, uses your mobile device’s GPS, gyroscope, and camera to create a custom 3D augmented reality experience which positions unreleased audio of the new album on the sun itself. Initially, you will hear static but as the sun nears dawn, the receiver will tune into today’s song until it is crystal clear. If the sun is behind you, it will sound like the song is coming from behind you. Once you’ve faced the dawn and heard the new clip, you’ll have the opportunity to take a custom photo, saving the moment. As the sun rises, the song will tune out and the static will tune back in. This functionality is no longer available at www.twilightreceiver.com but check out this CodePen prototype and read on to understand a bit more about how it functioned.
Forecasting Dawn
So, when is dawn exactly? Well, it is defined as the period when the sun is between 18° below the horizon to the horizon itself. Within this period are three types of dawn, starting from night: Astronomical dawn, Nautical dawn, and Civil dawn. I chose to focus on Civil dawn, as that’s when the sun is rising within 6° of the horizon until sunrise itself. SunCalc allows us to get these times by simply passing in a date and a pair of coordinates.
const sunTimes = SunCalc.getTimes(new Date(), latitude, longitude)
This will return many different sunlight times, including the two we are most interested in: dawn and sunrise. The dawn time represents when morning nautical twilight ends and morning civil twilight starts, aka civil dawn. The sunrise time represents when the top edge of the sun appears on the horizon, aka sunrise. If a user’s current time falls between these two times, it is currently dawn for them. I like using the date-fns date utility library to handle these comparisons easily. In particular, the isAfter and isBefore helpers are quite handy.
if (isAfter(Date.now(), sunTimes.dawn) && isBefore(Date.now(), sunTimes.sunrise)) {
// dawn
}Later on you’ll see that I’m also interested in the 6° below dawn and 6° above sunrise. These areas will be used to fade audio in and out.
Initializing Positional Audio
Releasing audio based on the sun’s altitude is pretty cool but what would make this even better is if the audio sounded like it was coming from the sun. In order to accomplish this in Three.js, we can define an AudioListener that represents our user’s location and a PositionalAudio instance which we can later associate with the 3D position of the sun.
Audio Listener
The AudioListener represents our user’s location so our app can properly interpret positional audio in the scene. We’ll initialize the listener and then add it to our camera, which represents our user’s point of view.
// Initialize audio listener
const listener = new THREE.AudioListener()
// Add listener to camera
camera.add(listener)
Now we can add our audio.
Positional Audio
PositionalAudio uses the Web Audio API to virtually position audio in 3D space relative to our listener. First, we must load our sound with an AudioLoader. Then, we can initialize the positioned audio using the loaded buffer. For our app, we’ll want the audio to continually loop so we can set that here. In addition, the volume will be set from a dynamic variable based on the sun’s altitude which I will discuss later. Finally, we’ll use setRefDistance to set how far the sound is traveling before it begins to reduce. In this case, the same distance our 3D positioned sun is from the user.
// Load position audio
const songBuffer = new THREE.AudioLoader().load('song.mp3')
// Initialize positional audio
const songAudio = new THREE.PositionalAudio(listener)
// Set buffer
songAudio.setBuffer(songBuffer)
// Set panner ref distance
songAudio.setRefDistance(10)
// Loop audio
songAudio.setLoop(true)
// Set audio volume
songAudio.setVolume(volume)
Now, let’s position our sun in the scene.
Positioning the Sun
We’re going to be using the getPosition method of SunCalc to position the sun in our scene. However, Volodymyr shared this 900 byte script he wrote on Observable which can accomplish the same thing. I’ll certainly be looking into this for future projects!
Add the Sun
The sun in our scene is just a Sprite instance from Three.js and I’m just using this simple png radial as the SpriteMaterial texture. Let’s load up the texture and initialize our sun sprite.
// Load sun texture
const sunTexture = new TextureLoader().load('sun.png')
// Initialize sun material
const sunMaterial = new THREE.SpriteMaterial({
map: sunTexture
})
// Initialize sun
const sun = new THREE.Sprite(sunMaterial)
// Add sun to scene
scene.add(sun)
Once the sun is initialized, we can then add our positioned song audio to it.
// Add song audio to sun
sun.add(songAudio)
Now, let’s position our sun.
Position the Sun
We can now use that SunCalc getPosition method to receive the altitude and azimuth of the sun at the current moment in time. Then those measurements can be converted into a pair of spherical coordinates which represent the 3D position of the sun in our scene.
// Sun positioning
let positionSun = () => {
// Get sun position from suncalc
let sunPosition = SunCalc.getPosition(Date.now(), latitude, longitude)
// Set vector 3 from altitude and azimuth
let sunVector = new THREE.Vector3().setFromSphericalCoords(10, sunPosition.altitude - THREE.Math.degToRad(90), -sunPosition.azimuth - THREE.Math.degToRad(180))
// Position sun
sun.position.copy(sunVector)
// Look at user
sun.lookAt(new THREE.Vector3())
}
Since we’ve added our song audio to the sun, as it repositions itself, so will the audio. This method can be called every second in our render loop to keep the sun correctly positioned.
Adjusting Volume
The final piece of the equation is adjusting the volume of our song audio based on the position of the sun. To do this, I will focus solely on some of the sunlight times the getTimes method of SunCalc provides. We already know we want the volume to be at 100% or 1 during dawn. In addition, we would like the volume to go from 0 to 1 within the 6° leading up to dawn and then from 1 to 0 within the 6° preceding dawn.
Let’s focus on those 6° before dawn. SunCalc defines -12° as nauticalDawn. Knowing this time, the time of dawn, and the user’s current time, we can compute a 0 to 1 ratio to adjust the volume. The equation is (currentTime — nauticalDawnTime) / (dawnTime — nauticalDawnTime). Since the 6° after dawn is inverted, we’ll reverse the equation a bit and use the sunrise and goldenHourEnd times instead.
volume() {
// Check times
if (Date.now() < sunTimes.sunrise && Date.now() > sunTimes.dawn) {
// Dawn
return 1 } else if (Date.now() < sunTimes.dawn && Date.now() > sunTimes.nauticalDawn) {
// Before Dawn
return (Date.now() - sunTimes.nauticalDawn.getTime()) / (sunTimes.dawn.getTime() - sunTimes.nauticalDawn.getTime()) } else if (Date.now() > sunTimes.sunrise && this.now < sunTimes.goldenHourEnd) {
// After Dawn
return 1 - ((Date.now() - sunTimes.sunrise.getTime()) / (sunTimes.goldenHourEnd.getTime() - sunTimes.sunrise.getTime())) } else {
// Any other time
return 0}
}
With this new computed volume property, we can dynamically adjust the song’s volume.
// Adjust song audio
songAudio.setVolume(volume)
In order to make this even more interesting, we added a second PositionalAudio of radio static which uses the inverse volume. When the song is at 0% volume, the static will be at 100%. This allows the receiver to simulate a FM radio of sorts, allowing the static to tune out and song to tune in based on the sun’s position.
Controls
The CodePen example of this technique uses OrbitControls to navigate the scene. However, the final experience used DeviceMotionControls so the user could use their mobile device’s orientation to control the camera, like any sort of AR application. For more information on how I handle that tech in Three.js, check out my recent Eddie Vedder case study.
Thanks
I couldn’t have built this without the support of Third Man Records, The Orchard, and, of course, Volodymyr. Jack White’s new album Fear of the Dawn is out now. The Twilight Receiver will continue to evolve to bridge the gap between this record and Entering Heaven Alive. Stay tuned. For more information on how you can stand with Ukraine, check out the following link.
Using Three.js to Hear the Dawn with Jack White was originally published in Bits and Pieces on Medium, where people are continuing the conversation by highlighting and responding to this story.
This content originally appeared on Bits and Pieces - Medium and was authored by Lee Martin
Lee Martin | Sciencx (2022-04-13T08:41:03+00:00) Using Three.js to Hear the Dawn with Jack White. Retrieved from https://www.scien.cx/2022/04/13/using-three-js-to-hear-the-dawn-with-jack-white/
Please log in to upload a file.
There are no updates yet.
Click the Upload button above to add an update.