I work for a performing arts institution and have been asked to look into incorporating wearable technology into accessibility for our patrons. I am interested in finding out more information regarding the use of SmartEyeglasses for supertitles (aka, subtitles) in live or pre-recorded performance. Is it possible to program several glasses to show the user(s) the same supertitles at the same time? How does this programming process work? Can several pairs of SmartEyeglasses connect with the same host device?
Any information is very much appreciated. I look forward to hearing from you!
Your question is overly broad and liable to be closed as such, but I'll bite:
The documentation for the SDK is available here: https://developer.sony.com/develop/wearables/smarteyeglass-sdk/api-overview/ - it describes itself as being based on Android's. The content of the wearable display is defined in a "card" (an Android UI concept: https://developer.android.com/training/material/lists-cards.html ) and the software runs locally on the glasses.
Things like subtitles for prerecorded and pre-scripted live performances could be stored using file formats like .srt ( http://www.matroska.org/technical/specs/subtitles/srt.html ) which are easy to work with and already have a large ecosystem around them, such as freely available tools to create them and software libraries to read them.
Building such a system seems simple then: each performance has an .srt file stored on a webserver somewhere. The user selects the performance somehow, and you'd write software which reads the .srt file and displays text on the Card based on the current timecode through until the end of the script.
...this approach has the advantage of keeping server-side requirements to a minimum (just a static webserver will do).
If you have more complex requirements, such as live transcribing, support for interruptions and unscripted events then you'd have to write a custom server which sends "live" subtitles to the glasses, presumably over TCP, this would drain the device's battery life as the Wi-Fi radio would be active for much longer. An alternative might be to consider Bluetooth, but I don't know how you'd build a system that can handle 100+ simultaneous long-range Bluetooth connections.
A compromise is to use .srt files, but have the glasses poll the server every 30 seconds or so to check for any unscripted events. How you handle this is up to you.
(As an aside, this looks like a fun project - please contact me if you're looking to hire someone to build it :D)
Each phone can only host only 1 SmartEyeglass. So you would need separate host phones for each SmartEyeglass.
Related
Preamble:
Recently I came across an interesting story about people who seem to be sending emails with documents that contain child pornography. This is an example (this one is jpeg but im hearing about it being done with PDFs, which generally cant be previewed)
https://www.youtube.com/watch?v=zislzpkpvZc
This can pose a real threat to people in investigative journalism, because even if you delete the file after its been opened in Temp the file may still be recovered by forensics software. Even just having opened the file already puts you in the realm of committing a felony.
This also can pose a real problem to security consultants for a group. Lets say person A emails criminal files, person B is suspicious of email and forwards it to security manager for their program. In order to analyze the file the consultant may have to download it on a harddrive, even if they load it in a VM or Sandbox. Even if they figure out what it is they are still in this legal landmine area that bad timing could land them in jail for 20 years. Thinking about this if the memory was to only enter the RAM then upon a power down all traces of this opened file would disappear.
Question: I have an OK understanding about how computer architecture works, but this problem presented earlier made me start wondering. Is there a limitation, at the OS, hardware, or firmware level, that prevents a program from opening a stream of downloading information directly to the RAM? If not let's say you try to open a pdf, is it possible for the file it's opening to instead be passed to the program as a stream of downloading bytes that could then rewrite/otherwise make retention of the final file on the hdd impossible?
Unfortunately I can only give a Linux/Unix based answer to this, but hopefully it is helpful and extends to Windows too.
There are many ways to pass data between programs without writing to the hard disk, it is usually more of a question of whether the software applications support it (web browser and pdf reader for your example). Streams can be passed via pipes and sockets, but the problem here is that it may be more convenient for the receiving program to seek back in the stream at certain points rather than store all the data in memory. This may be a more efficient use of resources too. Hence many programs do not do this. Indeed a pipe can be made to look like a file, but if the application tries to seek backward, it will cause an error.
If there was more demand for streaming data to applications, it would probably be seen in more cases though as there are no major barriers. Currently it is more common just to store pdfs in a temporary file if they are viewed in a plugin and not downloaded. Video can be different though.
An alternative is to use a RAM drive, it is common for a Linux system to have at least one set up by default (tmpfs), although it seems for Windows that you have to install additional software. Using one of these removes the above limitations and it is fairly easy to set a web browser to use it for temporary files.
I am creating an application that will pre-record a user's voice for each letter on the keyboard and when the app is running, if the user calls out '5', the system types 5 to which ever application is capable of accepting the input at that time. I am .NET person and venturing into XCode.
I have done some research and I am pretty sure of using AV Foundation for recording the audio. The question is how to use speech recognition in OSX and use it to identify a particular key on the keyboard...Will highly appreciate any feedback even if it might be general advice for the approach that I should take to tackle this project!
ThankS IN ADVANCE :) !
Let me be clear first. I have never done this before, but i have a general idea of how it is done. You need to Bind a audio file to a certain number/Key. Whenever a user speaks into the mic, you record their voice and upload it to a server, which compares the Audio File from the User to the pre recorded audio file the user made.
Here is a SO Question that talks about Audio Fingerprinting.
How can I Compare 2 Audio Files Programmatically?
You can compare the audio files in PHP/Python, and have it return a value. For example. If audio file a.mp3 (on server) matches to the newRecorded.mp3 the user just recorded, return a.mp3, then just strip the .mp3 and keep the key.
As far as recording sentences and commands, you might be able to do the same. I will continue to do more research on this and help you out as much as i can.
Hopefully this gives you a better idea and easier way of doing things.
Also there is this
https://developer.apple.com/library/mac/documentation/cocoa/reference/ApplicationKit/Classes/NSSpeechRecognizer_Class/Reference/Reference.html
and
https://developer.apple.com/library/mac/documentation/cocoa/conceptual/speech/Articles/RecognizeSpeech.html#//apple_ref/doc/uid/20002081-BCIHEBFH
This could be really helpful and would use built in speech recognition.
For a some-what small (at least hopefully) project, I am hoping to gain access to the current audio being played through the "main line" (i.e. what is heard through the speakers.) Specifically, I'd like to create a visual equalizer of the audio currently being played. I do not wish to capture or "tamper" with the audio in any way, just run a little analysis on it. That being said, I'd imagine access to such information is not handed out nicely in a high-level API.
I noticed a similar question which is concerned with looking at system sound. The accepted answer points to looking into Soundflower's source code. I am not completely adverse to doing this but I'd like to ensure there isn't a simpler way before I got into it (especially because I have no real audio programming experience, especially at the system level.)
Any input is very much appreciated,
--Sam
There is no simple way to do this on OS X. You really have to do this from a kext, unfortunately.
For my university I (and three others), are searching for a project that utilizes at least one embedded device, web services or other web technology, and a Graphical User Interface.
Currently we are looking at developing a unified remote, that is an extendable application on a cell phone through which you can control your media center. Any ideas, or advice on this will be appreciated, though it is not the focus of this question.
We are having a hard time finding interesting (or funny) projects on which we can work a complete semester. Any ideas will be greatly appreciated. The software will be released as free software. (GPL or BSD license).
We all have a Bsc in Software Engineering.
EDIT: I am very pleased with the suggestions so far. Thanks to everyone, and keep it coming.
How about follower: carry a device, as you move from room to room in your house devices configure themselves to your preference - lights, music etc. If two people are in the room some precedence rules.
Is that possible just on the presence of a mobile phone?
Another idea (from the top of my head):
A work environment ensurance thing. We programmers like to develop in nice and quiet environments. Unfortunately some people tends to annoy us with their disturbing behaviour (or just by being loud).
So the project could be to create devices wich tracks the stress level (sweat levels, pulse etc.) of the individual and their impact onto others.
An example: One individual is very loud (the device should measure this), and others around him becomes stressed and/or unfocused because of this. The serverside sw, should then detect and warn him to quit down a bit to improve the work environment.
Comments?
What do you peeps like doing? Build an app for it.
So, if you like drinking coffee build a application which will find the nearest frothy coffee shoppe (or if you're particular, the nearest Peets/Starbucks/Whatever-ocino). This idea works for beer too.
If you buy stuff off e-Bay build a sniper app.
If you enjoy playing frisbee build an app which locates your nearest friends and sends them a text asking whether they want to goof off lectures and go to the park.
Heck, you could even build an app which monitors your SO questions and alerts you when you get an answer (although I don't know whether the data services SO currently offer will be up to the job).
The standout companies that have made great universal (programmable) remotes are : logitech, and philips.
One of the big problems with these types of devices is the ability of the general consumer to actually program all of their various devices. Logitech has done an outstanding job of providing a fairly simple Web based user setup experience that then implements a very usable universal control.
I would definitely look at what they have done for some ideas on universal remote controls.
How about an app and hardware that will tell me when my wife's plants need watering? (It's somehow my fault if they don't get watered.)
OK then: the recipe generating fridge. Rfid tags on the contents know what's available and the expiry dates. The database knows the recipes. The fridge emails/texts you to say "buy some mushrooms and you can have a delicous ham and mushroom omelette while the eggs are still fresh."
Benjamin and all those aspiring to do embedded projects ...
When you start a project, especially in embedded systems, you need to understand that the hardware is not your PC but some special device. And every sensor will be a transducer in itself. The only thing that would matter to students is that everything costs and are costly
So, it will be good to make sure that the idea is such that,
It can be completed by the
project members within the given timeframe
All the required development
tools like hardware etc can be
really bought
Of all, it good to ensure that the
project enables you to learn
something useful for your career ...
To do all this it is better set some achievable goals
Develop a system in which you can program the lighting system of your house. You can set up their schedule one time and everything should work automatically.
I really love working witht the Atmel ststk1000/stk1006/stk1002 development boards for tht AVR32. ATSTK1000
2x Ethernet
QVGA lcd
USB 2.0
SD/MMC
Conpact flash
Supported embedded linux
IR
Audio
ps2 interfaces
uarts
++
familiy atmel page:
AVR 32 family home
online forums
Forums for CPU
Two years ago, we shipped a multi-gigabyte Windows application, with lots of video files. Now we're looking to release a significant update, with approximately 1 gigabyte of new and changed data.
We're currently looking at DVD fulfillment houses (like these folks, for example), which claim to be able to ship DVDs to our customers for $5 and up. Does anyone have any experience with these companies?
We've also looked at a bunch of network-based "updater" software. Unfortunately, most of these tools are intended for much smaller programs. Are there any libraries or products which handle gigabyte-sized updates well?
Thank you for your advice!
BITS is a library from Microsoft for downloading files piece by piece using unused bandwidth. You can basically have your clients trickle-download the new video files. The problem, however, is that you'll have to update your program to utilize BITS first.
Depending on who the end user is you have a few options:
Shipping DVD's
This option tends to be rather expensive, and may not be the best way, what if you are shipping it to someone that no longer has the software installed.
HTTP hosting (using Akamai, or any other CDN)
This works rather well for other companies, for example Apple and I believe Microsoft as well.
Bittorrent
It is not just used for illegal content, it will allow you to offload some of the work load of sending the file, and at the same time it is a fast protocol, if you make sure the that the machine seeding has the correct file, the bittorrent protocol will make sure the end user gets the same file with the exact same hash.
You can use the rsync algorithm: http://samba.anu.edu.au/rsync/