Game Audio Programming

Principles and Practices

Guy Somberg, Guy Somberg

• Publisher: CRC Press
• Year: 2016
• ISBN:
  9781315351650
• Edition: 1
• Topic: Computer Science
• Subtopic: Programming Games
• Index: Computer Science

Welcome to Game Audio Programming: Principles and Practices! This book is the first of its kind: an entire book dedicated to the art of game audio programming. With over fifteen chapters written by some of the top game audio programmers and sound designers in the industry, this book contains more knowledge and wisdom about game audio programming than any other volume in history.

One of the goals of this book is to raise the general level of game audio programming expertise, so it is written in a manner that is accessible to beginners, while still providing valuable content for more advanced game audio programmers. Each chapter contains techniques that the authors have used in shipping games, with plenty of code examples and diagrams. There are chapters on the fundamentals of audio representation and perception; advanced usage of several different audio middleware platforms (Audiokinetic Wwise, CRI ADX2, and FMOD Studio); advanced topics including Open Sound Control, Vector-Based Amplitude Panning, and Dynamic Game Data; and more!

Whether you're an audio programmer looking for new techniques, an up-and-coming game developer looking for an area to focus on, or just the one who got saddled with the audio code, this book has something for you.

• Cutting-edge advanced game audio programming concepts, with examples from real games and audio engines
• Includes perspectives of both audio programmers and sound designers on working and communicating together

• 
  
  

• Coverage not just on game audio engine design, but also on implementing audio tools and working with sound designers providing a comprehensive perspective on being an audio programmer

• 
  
  

IV

Advanced Topics

CHAPTER 10

Open Sound Control

Jorge Garcia

CONTENTS

10.1 Introduction

10.2 Why Integrate OSC in a Game Engine

10.3 Protocol Implementation

10.3.1 Client and Server Models

10.3.2 OSC Addresses, Messages, Bundles, and Packets

10.4 Dealing with Timing and Latency

10.5 Optimizations and Best Practices

10.6 Testing Approaches

10.7 Available Open Source OSC Libraries

10.8 Conclusion

Notes

10.1 INTRODUCTION

Open Sound Control (OSC) is a data definition in a way similar to XML or JSON. It was specifically designed and optimized to address some of the challenges related with sending control data between audio applications in the same machine, or between machines over a network.¹ OSC was originally created in 2009 at the University of California–Berkeley Center for New Music and Audio Technology for providing flexibility in a wide range of possible applications, from gestural control devices to audio systems integration and music synthesizers. The popularity of the implementations of OSC as a communication protocol has also found applications in more general networking scenarios beyond audio, in part because of its ease of use and implementation in a wide range of programming languages and hardware.²

If you are familiar with some audio equipment and musical instruments, you may be wondering what the differences are between the musical instrument digital interface (MIDI) protocol and OSC, and why OSC exists at all since MIDI seems to be capable of similar functionality. The truth is that this has been an ongoing debate in the music technology circles for a while, but I would say that OSC is a more general solution to the problems that we can find in audio production, and more particularly in scenarios that involve networked devices, although it’s also possible to stream MIDI data over a network (by using RTP-MIDI). It is true that recent proposals for a MIDI HD standard would overcome some of the limitations of MIDI, but OSC has been around for a while already and its maturity and simplicity makes it still ideal for some of the use cases that we can find in game audio development. One of the advantages of OSC is that it allows us to model any data format and meaning that we want, as we are not constrained with the typical musical notation and control change parameters that are found in the MIDI protocol. So the abstractions built with OSC can go beyond music terminology and are customizable for being easily human-readable, as we will see in this chapter.

I will start by explaining the value of integrating OSC in a game engine and a toolset, and the problems it can solve for both sound designers and audio programmers. Then we will go into some of the specifics and examples of a typical OSC implementation over UDP. I will also cover some of the possible optimizations that we may want to carry out for taking the most of the protocol. Some of the testing approaches and actual use cases are mentioned. This chapter then wraps up by mentioning some open source implementations of OSC that are freely available and that you can use straight from the box in case you don’t need to roll out your own low-level implementation.

10.2 WHY INTEGRATE OSC IN A GAME ENGINE

So, we have a flexible networking protocol at our disposal—why should we be using it? There are some straightforward answers to this question if we think about it as a technical solution for communicating a toolset with the game runtime. But in the end, the reasons would depend on the flexibility of your in-house game audio engine (or the middleware you are using), and the design approach of the sound designers you are working with. If we think about the possibilities of real-time tweaking for game audio parameters, probably OSC can be a clear winner in order to reduce the iteration time spent from designers and for expanding their creativity. And not only that: being able to easily integrate gestural controllers, EEG and VR headsets, networked musical equipment, or mobile devices with your game engine opens the doors for endless play and experimentation in the prototyping and incubation stages of a project. This would require a varying effort at the beginning depending on what you want to achieve, but as we will see, OSC is in the end a simple data definition and worth integrating as a protocol in your engine even if you have to roll out your own low-level implementation. This will allow you to “glue” different applications and devices easily by just using control messages. As an example, Guerrilla Games used it in their toolset.³ Additional and upcoming uses of OSC also cover the control of Procedural Audio and synthesis models as well as collaboration in content creation scenarios.⁴

10.3 PROTOCOL IMPLEMENTATION

Before entering into any code details, it’s good to have a bit of understanding of how networking protocols work. You have probably heard of UDP and TCP. The former is a popular protocol used in online games because it doesn’t require checking the integrity of the data we are sending (usually in packets), so it inherently provides low latency, which is preferred for having a smooth and responsive communication stream. On the other hand, TCP is more popular in regular internet scenarios where the guarantee of receiving the data correctly (and in the correct order) is more important than latency. I’m not going to enter into the details of how UDP and TCP work since this goes beyond the scope of this chapter and this book, but you would be able to learn more about it from some standard computer networking literature. What we need to know is that for most of the scenarios, if we desire real-time low latency (within the range of 15 ms and below), we would initially want to start with UDP for generating a stream of data. This will also allow us to skip the hassle of dealing with some session setup and handling, and just start streaming messages from a sender (a “client” as defined in the OSC specification⁵) to a listener (“server” that digests and processes incoming messages).

10.3.1 Client and Server Models

So, we have this server and client model, but how does this correspond to my tools, to my game engine, or the mobile phone that I want to use for controlling the parameters in my game? Again, it depends on what w...

Table of contents

• Cover
• Half Title
• Title Page
• Copyright Page
• Dedication
• Table of Contents
• Preface
• Acknowledgments
• Editor
• Contributors
• SECTION I FUNDAMENTALS
• SECTION II MIDDLEWARE
• SECTION III SOUND DESIGNER PERSPECTIVES
• SECTION IV ADVANCED TOPICS
• INDEX