WIP: Geometry Nodes: Scene Sound Input Node #107007

The objective of this potential patch is to provide geometry node users with the ability to make their geometry respond to the currently playing sound. This is accomplished by providing essential information such as the current sound volume and pitch.

Status: On Halt, Awaiting Design Approval

In order to make this work i will need to reach for a sound module developer to find the proper approach to evaluate rendered sound information ( ideally , at designated frame ).

Advantages

Working with the sound generated by blender directly (instead of importing and trying to synchronize a new sound data with timeline) is intuitive and easy to use for the user.

Use Case

This functionality can be used to create a range of effects where a geometry would react to a sound, such as:

Room objects reacting to earthquake sounds
Curve lighting strikes reacting to thunder
Approximate mouth movements.
Sound reacting abstract or text Mograph's

The concept can be tested already, as it was first implemented from a plugin that could read blender sequencer sound. ( Note that the goal of this patch is to read the generic sound blender is rendering, not only sounds from the sequencer )

The objective of this potential patch is to provide geometry node users with the ability to make their geometry respond to the currently playing sound. This is accomplished by providing essential information such as the current sound volume and pitch. ### Status: On Halt, Awaiting Design Approval In order to make this work i will need to reach for a sound module developer to find the proper approach to evaluate rendered sound information ( ideally , at designated frame ). ### Advantages Working with the sound generated by blender directly (instead of importing and trying to synchronize a new sound data with timeline) is intuitive and easy to use for the user. ### Use Case This functionality can be used to create a range of effects where a geometry would react to a sound, such as: - Room objects reacting to earthquake sounds - Curve lighting strikes reacting to thunder - Approximate mouth movements. - Sound reacting abstract or text Mograph's <img src="https://projects.blender.org/blender/blender/issues/107007/attachments/ab15ec89-23ba-429b-bd8e-7f2d387ce76d"> The concept can be tested already, as it was first implemented from a [plugin that could read blender sequencer sound](https://devtalk.blender.org/t/extra-nodes-for-geometrynodes/20942). ( Note that the goal of this patch is to read the generic sound blender is rendering, not only sounds from the sequencer ) <video src="https://projects.blender.org/blender/blender/issues/107007/attachments/01c1b2bb-5db5-4d7d-95aa-a10bbfb04c0c" controls>

dVByWlCJ9cIHnFte.mp4

355 KiB

thumb-Screenshot 2023-04-16 135056.jpg

12 KiB

Hi, it would be useful to have volume output with a -1 to 1 range. To simulate a speaker diaphragm you need positive and negative "voltage". Also this would allow for oscilloscopes to be made. Thanks for your consideration, cheers

Hi, it would be useful to have volume output with a -1 to 1 range. To simulate a speaker diaphragm you need positive and negative "voltage". Also this would allow for oscilloscopes to be made. Thanks for your consideration, cheers

First the design of fetching blender rendered sound need to be approved, then indeed, I could try to investigate the best information users might need and contact a sound module developer for advices on how to get such informations

> Hi, it would be useful to have volume output with a -1 to 1 range. To simulate a speaker diaphragm you need positive and negative "voltage". Also this would allow for oscilloscopes to be made. Thanks for your consideration, cheers First the design of fetching blender rendered sound need to be approved, then indeed, I could try to investigate the best information users might need and contact a sound module developer for advices on how to get such informations

Built in filtering would be nice as well, so you can easily select bands of different frequencies. Also I remember that other similar implementations I've tried for Animation Nodes for example had no way of "normalizing" the frequencies either, so in the end it was just the bass that contributed any energy while the upper frequencies almost did nothing. Which is how sounds work, but it's hard to do things like a visualizer this way :)

Download

What's New

Blender Studio

Manual

Developers Blog

Documentation

Benchmark

Blender Conference

Development Fund

One-time Donations

WIP: Geometry Nodes: Scene Sound Input Node #107007

Status: On Halt, Awaiting Design Approval

Advantages

Use Case

Reviewers