Job Posting: Senior VR/XR & Computer Vision Software Engineer With Experience Building Full-Stack AI-Driven Immersive Systems
Location: Remote (AWS-Cloud Focused)
Infrastructure: Ubuntu 24.04, AWS (G5/C6g), Red5 Pro (Streaming Server, TrueTime, and Terraform), NGINX
Stack: TensorFlow, Three.js / Babylon.js, React / Vue.js / Django, Node.js
What You’ll Build & Prototype
AI‑Driven Computer Vision & Facial Tracking
• Rapidly prototype and refine facial landmark detection pipelines using MediaPipe, ARKit/ARCore, or custom TensorFlow/PyTorch models.
• Build ML‑based blendshape mapping systems (regression, PCA, neural networks) based on live-streaming data that drive expressive 2D and 3D full-body representation of a person.
• Experiment with new Cross-Validation/Machine Learning techniques and integrate them into real‑time pipelines with sub‑100‑ms latency.
• Use AI tools to accelerate model iteration, data labeling, and performance tuning.
• AI Integration, implementation, and development of fine-tune tracking models using Google MediaPipe, TensorFlow, ARKit, and NVIDIA Maxine.
3D Graphics, Avatar Animation & Blender Integration
• Build expressive 3D environments and animation systems using Three.js.
• Work directly with Blender to create, modify, and export glTF avatars with morph targets and PBR materials.
• Prototype new animation techniques, shader effects, and environment interactions quickly.
• Shader Programming by writing custom GLSL/HLSL shaders to handle realistic skin textures, PBR materials, and real-time lighting within the virtual space.
• 3D Environment Development: Architect and render realistic 3D worlds using Three.js, Babylon.js, and Blender ensuring high-performance execution via WebGL and WebGPU.
• Collaborate with artists and designers in a fast feedback loop.
Real‑Time WebRTC & Distributed Media Systems
• Architect WebRTC pipelines for video, audio, and data channels.
• Integrate with our AWS‑hosted Red5 Pro cluster, including Streaming Manager, TrueTime Meeting, Terraform, and autoscaling components.
• Prototype new networking strategies, optimize ICE/TURN/STUN flows, and experiment with adaptive bitrate logic.
• Use AI‑assisted debugging and monitoring tools to accelerate iteration.
• Develop live real-time avatars and optimize the pipeline for mapping 2D webcam/camera feeds to 3D meshes using Facial Action Coding System (FACS) and morph target influences.
Startup‑Style Collaboration
• Work closely with founders and cross‑functional teammates to shape product direction.
• Prototype features rapidly, test them with real users, and iterate based on feedback.
• Contribute to architectural decisions, technical strategy, and long‑term platform vision.
• Thrive in a culture that values creativity, autonomy, and speed.
Role Overview
We are seeking a high-caliber Senior Full-Stack Software Engineer/Developer with a specialization in VR/XR and Computer Vision to lead the engineering of our "metaverse-class" social immersion platform. This role is at the bleeding edge of technology: you will build the systems that overlay live-streamed facial video onto computer-generated and live-captured 2D and 3D images of a person, creating a seamless "hybrid" user representation in realistic 3D worlds.
As a Full-Stack lead, you will own the entire vertical—from the Ubuntu/AWS backend and Red5 Pro streaming pipelines to the TensorFlow tracking models and the React/Three.js frontend environments.
1. Full-Stack Responsibilities & Frameworks
· Frontend (Web & VR/XR): Develop high-performance, immersive interfaces using React or Vue.js. Implement the 3D world layer using Three.js or Babylon.js, ensuring the hybrid-user representation renders with sub-100ms latency via WebGL.
· Backend & API: Architect robust server-side logic using Django or Node.js to manage user sessions, world state synchronization, and secure data persistence, Red5 Pro streaming server, NGINX (reverse proxy, SSL, caching), Docker/Kubernetes (scaling), and Redis/PostgreSQL for session/state management.
· Streaming Infrastructure: Manage the Red5 Pro TrueTime Meeting and Stream Manager deployment on Ubuntu 24.04, handling the ingestion and distribution of ultra-low latency WebRTC streams.
· Native Integration: Maintain and extend client functionality for Windows and Mobile using C# or C++.
2. AI, Computer Vision & Prototyping
· CV/ML Pipeline: Develop and integrate TensorFlow (and TensorFlow.js) models for real-time facial landmarking. Use Cross-Validation to ensure tracking accuracy across all user demographics.
· AI-Expedited Dev: Utilize AI tools (GitHub Copilot, Gemini, Cursor, v0.dev, etc.) to accelerate full-stack feature delivery and Luma AI for rapid 3D environment asset generation.
· Hybrid Rendering: Write custom GLSL/HLSL shaders to blend live video textures onto 2D and 3D meshes realistically.
3. Methodology & Security (Secure SDLC)
· Secure SDLC: Apply a "Security by Design" philosophy across the full stack or critical components of the stack. Perform threat modeling on WebRTC signaling, implement secure OAuth2/OIDC flows, and manage NGINX for SSL termination and WAF protection.
· Agile Methodology: Lead and participate in Agile workflows (Scrum/Kanban), translating complex user stories into technical requirements and iterative releases.
· DevOps: Orchestrate AWS infrastructure (EC2, S3, CloudFormation) and optimize the Ubuntu server environment for high-throughput media processing.
4. Technical Toolbox
Category
Requirements
Frontend Frameworks
React, Vue.js, Angular, Three.js, Babylon.js
Backend Frameworks
Django, Node.js, FastAPI
Streaming & Network
Red5 Pro (TrueTime & Streaming Manager API), WebRTC, RTSP, NGINX, Ubuntu 24.04
AI & CV
TensorFlow, OpenCV, MediaPipe, Scikit-learn
Cloud (AWS)
EC2 (G5/C6g), S3, Route53, AWS Wavelength (5G Edge)
Languages
Python, JavaScript/TypeScript, Java, C#, C++
Methodologies
Agile, Secure SSDLC
DevOps
AWS CloudFormation/Terraform
3D/Graphics
Three.js, GLSL Shaders, UV Mapping, Blender
Databases
Mongo, MySQL, NoSQL, Redis/PostgreSQL
Key Qualifications
· Experience: 7+ years in Full-Stack development, with at least 3 years focused on Real-Time Communications (RTC), VR/XR, WebRTC, and Computer Vision.
· Architecture: Proven ability to design and deploy scalable, secure client-server cloud-native applications on AWS, VPS, and bare-metal servers.
· Problem Solving: Deep understanding of the synchronization challenges between live video streams and 3D coordinate data.
· Security: Solid experience implementing security best practices throughout the software development life cycle.
· Portfolio demonstrating AI-driven video manipulation or Web3D immersive environments.
How to Apply
Submit your resume to
[email protected] with a link to your GitHub or portfolio. We are specifically looking for candidates who can demonstrate a completed project involving real-time video integration using Red5 Pro implementations on AWS platform, TensorFlow-based tracking, 3D web rendering, and a secure backend architecture.
Pay: $30.00 - $40.00 per hour
Work Location: Remote
Apply tot his job
Apply To this Job