Python Libtorrent: Pausing torrent not working as expected - libtorrent

I am trying to use Python-Libtorrent to download torrents. I have an issue where pausing the torrent doesn't work as expected:
import libtorrent as lt
import time
import sys
def get_libtorrent_session_and_start_dht(start_port, end_port):
ses = lt.session()
ses.listen_on(start_port, end_port)
ses.add_dht_router('dht.transmissionbt.com', start_port)
ses.add_dht_router('router.bittorrent.com', start_port)
ses.add_dht_router('router.utorrent.com', start_port)
ses.start_dht()
return ses
const_state_str = ['queued', 'checking', 'downloading metadata', \
'downloading', 'finished', 'seeding', 'allocating', 'checking fastresume']
const_pause_torrent = "pause_torrent"
const_resume_torrent = "resume_torrent"
const_kill_torrent = "kill_torrent"
const_user_logged_in = "user_logged_in"
const_user_logged_out = "user_logged_out"
const_quit_seeding = "quit_seeding"
ses = get_libtorrent_session_and_start_dht(6881, 6891)
torrent_info = None
torrent_handle = None
try:
torrent_info = lt.torrent_info("/home/horvste/Downloads/ubuntu-14.04.4-desktop-amd64.iso.torrent")
torrent_handle = ses.add_torrent(
{'ti': torrent_info, 'save_path': "/home/horvste/Downloads"})
except Exception as exception:
raise exception
while not torrent_handle.has_metadata():
print "Getting meta data"
time.sleep(1)
current_iteration = 0
while not torrent_handle.is_seed():
torrent_status = torrent_handle.status()
print "current_iteration: " + str(current_iteration)
if current_iteration == 10:
print "Calling torrent_handle.pause()...Pausing Torrent"
sys.stdout.flush()
torrent_handle.pause()
while not torrent_handle.status().paused:
print "Torrent Is Not Paused Yet"
time.sleep(1)
sys.stdout.flush()
while torrent_handle.status().paused:
print "Torrent Is Paused!"
torrent_handle.pause()
time.sleep(1)
sys.stdout.flush()
if const_state_str[torrent_status.state] == "checking":
print 'Checking Torrent....'
continue
print '\r%.2f%% complete (down: %.1f kb/s up: %.1f kB/s peers: %d) %s' % \
(torrent_status.progress * 100, torrent_status.download_rate / 1000, torrent_status.upload_rate / 1000, \
torrent_status.num_peers, const_state_str[torrent_status.state]), \
sys.stdout.flush()
current_iteration += 1
if torrent_status.paused:
print "Is Paused"
else:
print "Is Not Paused"
time.sleep(1)
Here is this scripts output:
...Previous Output Checking Torrent
Checking Torrent....
current_iteration: 0
4.17% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0) downloading None
Is Not Paused
current_iteration: 1
4.17% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 0) downloading None
Is Not Paused
current_iteration: 2
4.17% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 4) downloading None
Is Not Paused
current_iteration: 3
4.17% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 7) downloading None
Is Not Paused
current_iteration: 4
4.17% complete (down: 1.0 kb/s up: 1.0 kB/s peers: 11) downloading None
Is Not Paused
current_iteration: 5
4.17% complete (down: 4.0 kb/s up: 2.0 kB/s peers: 16) downloading None
Is Not Paused
current_iteration: 6
4.17% complete (down: 52.0 kb/s up: 5.0 kB/s peers: 21) downloading None
Is Not Paused
current_iteration: 7
4.17% complete (down: 132.0 kb/s up: 8.0 kB/s peers: 28) downloading None
Is Not Paused
current_iteration: 8
4.26% complete (down: 234.0 kb/s up: 12.0 kB/s peers: 33) downloading None
Is Not Paused
current_iteration: 9
4.31% complete (down: 317.0 kb/s up: 15.0 kB/s peers: 38) downloading None
Is Not Paused
current_iteration: 10
Calling torrent_handle.pause()...Pausing Torrent
Torrent Is Paused!
Torrent Is Paused!
Torrent Is Paused!
4.31% complete (down: 418.0 kb/s up: 19.0 kB/s peers: 46) downloading None
Is Not Paused
current_iteration: 11
4.44% complete (down: 0.0 kb/s up: 0.0 kB/s peers: 10) downloading None
Is Not Paused
current_iteration: 12
4.44% complete (down: 0.0 kb/s up: 1.0 kB/s peers: 12) downloading None
Is Not Paused
current_iteration: 13
4.44% complete (down: 1.0 kb/s up: 2.0 kB/s peers: 21) downloading None
...Torrent keeps downloading
As you can see from the above script, I call pause on the torrent handle and it pauses for 3 iterations of the loop and starts downloading again. It is important to note that the previous script did not have the torrent_handle.pause() call in this while loop:
while torrent_handle.status().paused:
print "Torrent Is Paused!"
torrent_handle.pause()
time.sleep(1)
sys.stdout.flush()
I am still getting the same output; the torrent doesn't pause as expected. I am running Ubuntu 14.04.4 and everything is installed via apt-get. In the libtorrent init.py my version number is version = '0.16.13.0'. Am I missing something or misusing the library?

Torrents are auto managed by default and that could bring them out of pause, check flags in add_torrent_params

Related

Elasticsearch stuck in yellow status

I fired the following command on elasticsearch
PUT /_cluster/settings
{
"persistent" : {
"threadpool.index.queue_size": -1
}
}
But now my elastuicsearch cluster health is stuck in yellow. Nothing is moving, and the cat recovery api gives following
books 0 613 replica done server1.internal.com server2.internal.com n/a n/a 1 100.0% 79 100.0%
books 0 53479 replica done server2.internal.com server3.internal.com n/a n/a 146 100.0% 435062890 100.0%
books 1 592 replica done server1.internal.com server2.internal.com n/a n/a 1 100.0% 79 100.0%
books 1 5901 replica done server2.internal.com server3.internal.com n/a n/a 198 34.3% 449403096 2.7%
books 2 551 replica done server1.internal.com server2.internal.com n/a n/a 1 100.0% 79 100.0%
books 2 9018 replica done server2.internal.com server3.internal.com n/a n/a 201 28.9% 451881473 4.4%
books 3 519 replica done server1.internal.com server2.internal.com n/a n/a 1 100.0% 79 100.0%
books 3 3869 replica done server2.internal.com server3.internal.com n/a n/a 170 61.8% 434156880 1.2%
books 4 525 replica done server1.internal.com server2.internal.com n/a n/a 1 100.0% 79 100.0%
books 4 33468 replica done server2.internal.com server3.internal.com n/a n/a 136 100.0% 428616146 100.0%
For about 30 minutes there has been no progress.
Can anyone help me how to solve this?
I just figured out this was due to corrupted disk on one of the nodes.
Classification of Elasticsearch Cluster Status:
RED: Some or all of (primary) shards are not ready.
YELLOW: Elasticsearch has allocated all of the primary shards, but some/all of the replicas have not been allocated.
GREEN: Cluster is fully operational. Elasticsearch is able to allocate all shards and replicas to machines within the cluster.

ffmpeg produced .wav reads only zeros with scipy.io.wavfile

Hi everyone and thanks for reading.
I wanted to do some analysis on a song using Python's scipy.io.wavfile. Since I only have the song as .mp3 I converted the file to .wav using ffmpeg the following way:
ffmpeg -i test.mp3 test.wav
The .wav file plays perfectly well with vlc player, but wavfile shows only zeroes when reading it:
from scipy.io import wavfile as wf
data = wf.read("test.wav")
C:\Program Files\Anaconda\lib\site-packages\scipy\io\wavfile.py:42: WavFileWarning: Unknown wave file format
warnings.warn("Unknown wave file format", WavFileWarning)
data
(44100, array([[0, 0],
[0, 0],
[0, 0],
...,
[0, 0],
[0, 0],
[0, 0]], dtype=int16))
I tried getting the data with Python's built-in wave module before to the same effect (only zeros).
I am using the 64bit version of ffmpeg (ffmpeg-20140218-git-61d5970-win64-static).
Any help is appreciated :-)
Edit: Included .wav header and tried forcing ffmpeg output format
I guess the header information of the .wav file is included here:
ffmpeg -i .\test.wav
Guessed Channel Layout for Input Stream #0.0 : stereo
Input #0, wav, from '.\test.wav':
Metadata:
artist : Joe Cocker
copyright : (C) 1987 Capitol Records, Inc.
date : 1987
genre : Pop
title : Unchain My Heart
album : Unchain My Heart
track : 1/10
encoder : Lavf55.33.100
Duration: 00:05:04.33, bitrate: 1411 kb/s
Stream #0:0: Audio: pcm_s16le ([1][0][0][0] / 0x0001), 44100 Hz, stereo, s16, 1411 kb/s
If I try to specify the ffmpeg output format explicitly for the .mp3 conversion:
ffmpeg -i .\test.mp3 -f s16le -ar 44100 -ac 2 test.wav
Input #0, mp3, from '.\test.mp3':
Metadata:
title : Unchain My Heart
artist : Joe Cocker
album : Unchain My Heart
genre : Pop
composer : Bobby Sharp
track : 1/10
disc : 1/1
album_artist : Joe Cocker
copyright : (C) 1987 Capitol Records, Inc.
date : 1987
Duration: 00:05:04.35, start: 0.025056, bitrate: 240 kb/s
Stream #0:0: Audio: mp3, 44100 Hz, stereo, s16p, 235 kb/s
Stream #0:1: Video: mjpeg, yuvj420p(pc), 600x600 [SAR 1:1 DAR 1:1], 90k tbr, 90k tbn, 90k tbc
Metadata:
title :
comment : Cover (front)
Output #0, s16le, to 'test.wav':
Metadata:
title : Unchain My Heart
artist : Joe Cocker
album : Unchain My Heart
genre : Pop
composer : Bobby Sharp
track : 1/10
disc : 1/1
album_artist : Joe Cocker
copyright : (C) 1987 Capitol Records, Inc.
date : 1987
encoder : Lavf55.33.100
Stream #0:0: Audio: pcm_s16le, 44100 Hz, stereo, s16, 1411 kb/s
Stream mapping:
Stream #0:0 -> #0:0 (mp3 -> pcm_s16le)
Press [q] to stop, [?] for help
video:0kB audio:52425kB subtitle:0 data:0 global headers:0kB muxing overhead 0.000000%
size= 52425kB time=00:05:04.32 bitrate=1411.2kbits/s
But in this case (forced format), both ffmpeg and wavfile are not able to read the file:
ffmpeg -i .\test.wav
.\test.wav: Invalid data found when processing input
and
data = wf.read("test2.wav")
---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
<ipython-input-10-fbbd84cb966b> in <module>()
----> 1 data = wf.read("test2.wav")
C:\Program Files\Anaconda\lib\site-packages\scipy\io\wavfile.pyc in read(filename, mmap)
152
153 try:
--> 154 fsize = _read_riff_chunk(fid)
155 noc = 1
156 bits = 8
C:\Program Files\Anaconda\lib\site-packages\scipy\io\wavfile.pyc in _read_riff_chunk(fid)
98 _big_endian = True
99 elif str1 != b'RIFF':
--> 100 raise ValueError("Not a WAV file.")
101 if _big_endian:
102 fmt = '>I'
ValueError: Not a WAV file.
I encountered the same problem. This seems to be a bug in FFmpeg that was introduced in October 2011 that was fixed April 29, 2014 (5e7d21c7ad02e37caa1bcb50ab8ad64e7d7fb86c). FFmpeg versions more recent than 2.3 (July 16, 2014) should write WAVs that numpy can read without error.

ffmpeg error when using non root user

I have installed ffmpeg and it works perfect if you are root or using sudo but it errors out when trying to use it as a non-root user. I have performed the same install on a test VPS and it installs without issue so it appears to be isolated to this cPanel server. I haven't been able to identify why.
Non-Root:
> ffmpeg -v debug -i 1.mov 1.avi ffmpeg version N-65949-g0ddb051
> Copyright (c) 2000-2014 the FFmpeg developers built on Aug 28 2014
> 11:39:47 with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-4)
> configuration: --enable-gpl --enable-libmp3lame --enable-libvorbis
> --enable-libvpx --enable-libx264 libavutil 54. 7.100 / 54. 7.100 libavcodec 56. 0.101 / 56. 0.101 libavformat 56. 3.100 / 56. 3.100 libavdevice 56. 0.100 / 56. 0.100 libavfilter 5. 0.103 / 5. 0.103 libswscale 3. 0.100 /
> 3. 0.100 libswresample 1. 1.100 / 1. 1.100 libpostproc 53. 0.100 / 53. 0.100 Splitting the commandline. Reading option '-v' ... matched as option 'v' (set logging level) with argument 'debug'.
> Reading option '-i' ... matched as input file with argument '1.mov'.
> Reading option '1.avi' ... matched as output file. Finished splitting
> the commandline. Parsing a group of options: global . Applying option
> v (set logging level) with argument debug. Successfully parsed a group
> of options. Parsing a group of options: input file 1.mov. Successfully
> parsed a group of options. Opening an input file: 1.mov.
> [mov,mp4,m4a,3gp,3g2,mj2 # 0x348fc20] Format mov,mp4,m4a,3gp,3g2,mj2
> probed with size=2048 and score=100 [mov,mp4,m4a,3gp,3g2,mj2 #
> 0x348fc20] ISO: File Type Major Brand: qt [mov,mp4,m4a,3gp,3g2,mj2 #
> 0x348fc20] Before avformat_find_stream_info() pos: 698348 bytes
> read:39602 seeks:1 [mov,mp4,m4a,3gp,3g2,mj2 # 0x348fc20] All info
> found [mov,mp4,m4a,3gp,3g2,mj2 # 0x348fc20] After
> avformat_find_stream_info() pos: 169190 bytes read:2333362 seeks:67
> frames:148 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mov':
> Metadata:
> major_brand : qt
> minor_version : 0
> compatible_brands: qt
> creation_time : 2014-06-05 04:40:27
> model : iPhone 5
> model-eng : iPhone 5
> encoder : 7.1.1
> encoder-eng : 7.1.1
> date : 2014-06-04T21:40:27-0700
> date-eng : 2014-06-04T21:40:27-0700
> make : Apple
> make-eng : Apple Duration: 00:00:07.15, start: 0.000023, bitrate: 781 kb/s
> Stream #0:0(und), 41, 1/600: Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m), 480x360, 1/1200, 710 kb/s, 29.09
> fps, 600 tbr, 600 tbn, 1200 tbc (default)
> Metadata:
> rotate : 90
> creation_time : 2014-06-05 04:40:27
> handler_name : Core Media Data Handler
> encoder : H.264
> Side data:
> displaymatrix: rotation of -90.00 degrees
> Stream #0:1(und), 107, 1/44100: Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 63 kb/s (default)
> Metadata:
> creation_time : 2014-06-05 04:40:27
> handler_name : Core Media Data Handler Successfully opened the file. Parsing a group of options: output file 1.avi. Successfully
> parsed a group of options. Opening an output file: 1.avi. Successfully
> opened the file. detected 24 logical cores [graph 0 input from stream
> 0:0 # 0x34aa3a0] Setting 'video_size' to value '480x360' [graph 0
> input from stream 0:0 # 0x34aa3a0] Setting 'pix_fmt' to value '0'
> [graph 0 input from stream 0:0 # 0x34aa3a0] Setting 'time_base' to
> value '1/600' [graph 0 input from stream 0:0 # 0x34aa3a0] Setting
> 'pixel_aspect' to value '0/1' [graph 0 input from stream 0:0 #
> 0x34aa3a0] Setting 'sws_param' to value 'flags=2' [graph 0 input from
> stream 0:0 # 0x34aa3a0] Setting 'frame_rate' to value '320/11' [graph
> 0 input from stream 0:0 # 0x34aa3a0] w:480 h:360 pixfmt:yuv420p
> tb:1/600 fr:320/11 sar:0/1 sws_param:flags=2 [format # 0x3496b00]
> compat: called with args=[yuv420p] [format # 0x3496b00] Setting
> 'pix_fmts' to value 'yuv420p' [AVFilterGraph # 0x348f320]
> query_formats: 4 queried, 3 merged, 0 already done, 0 delayed
> [AVFilterGraph # 0x34bbe60] Error initializing threading.
> [AVFilterGraph # 0x34bbe60] Error creating filter 'anull' Error
> opening filters! [AVIOContext # 0x34967c0] Statistics: 0 seeks, 0
> writeouts [AVIOContext # 0x348f1e0] Statistics: 2333362 bytes read, 67
> seeks
[AVFilterGraph # 0x34bbe60] Error initializing threading.
[AVFilterGraph # 0x34bbe60] Error creating filter 'anull' Error
opening filters!
As ROOT:
ffmpeg -v verbose -i 1.mov 1.avi
ffmpeg version N-65949-g0ddb051 Copyright (c) 2000-2014 the FFmpeg developers
built on Aug 28 2014 11:39:47 with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-4)
configuration: --enable-gpl --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264
libavutil 54. 7.100 / 54. 7.100
libavcodec 56. 0.101 / 56. 0.101
libavformat 56. 3.100 / 56. 3.100
libavdevice 56. 0.100 / 56. 0.100
libavfilter 5. 0.103 / 5. 0.103
libswscale 3. 0.100 / 3. 0.100
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 0.100 / 53. 0.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mov':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2014-06-05 04:40:27
model : iPhone 5
model-eng : iPhone 5
encoder : 7.1.1
encoder-eng : 7.1.1
date : 2014-06-04T21:40:27-0700
date-eng : 2014-06-04T21:40:27-0700
make : Apple
make-eng : Apple
Duration: 00:00:07.15, start: 0.000023, bitrate: 781 kb/s
Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m), 480x360, 710 kb/s, 29.09 fps, 600 tbr, 600 tbn, 1200 tbc (default)
Metadata:
rotate : 90
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
encoder : H.264
Side data:
displaymatrix: rotation of -90.00 degrees
Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 63 kb/s (default)
Metadata:
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
[graph 0 input from stream 0:0 # 0x27ed3a0] w:480 h:360 pixfmt:yuv420p tb:1/600 fr:320/11 sar:0/1 sws_param:flags=2
[graph 1 input from stream 0:1 # 0x2800700] tb:1/44100 samplefmt:fltp samplerate:44100 chlayout:0x4
Output #0, avi, to '1.avi':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
make-eng : Apple
model : iPhone 5
model-eng : iPhone 5
make : Apple
ISFT : Lavf56.3.100
ICRD : 2014-06-04T21:40:27-0700
date-eng : 2014-06-04T21:40:27-0700
Stream #0:0(und): Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 480x360, q=2-31, 200 kb/s, 29.09 fps, 29.09 tbn, 29.09 tbc (default)
Metadata:
rotate : 90
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
encoder : Lavc56.0.101 mpeg4
Stream #0:1(und): Audio: mp3 (libmp3lame) (U[0][0][0] / 0x0055), 44100 Hz, mono, fltp (default)
Metadata:
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
encoder : Lavc56.0.101 libmp3lame
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
Stream #0:1 -> #0:1 (aac (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
*** dropping frame 79 from stream 0 at ts 80
*** dropping frame 113 from stream 0 at ts 114
*** dropping frame 145 from stream 0 at ts 146
*** dropping frame 177 from stream 0 at ts 178
No more output streams to write to, finishing.
frame= 204 fps=0.0 q=27.2 Lsize= 348kB time=00:00:07.15 bitrate= 398.4kbits/s dup=0 drop=4
video:271kB audio:56kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 6.560985%
Input file #0 (1.mov):
Input stream #0:0 (video): 208 packets read (634624 bytes); 208 frames decoded;
Input stream #0:1 (audio): 308 packets read (56854 bytes); 308 frames decoded (315392 samples);
Total: 516 packets (691478 bytes) demuxed
Output file #0 (1.avi):
Output stream #0:0 (video): 204 frames encoded; 204 packets muxed (276993 bytes);
Output stream #0:1 (audio): 274 frames encoded (315392 samples); 275 packets muxed (57469 bytes);
Total: 479 packets (334462 bytes) muxed
strace and strace -e outputs:
strace non root:
579 clone(child_stack=0x7fd3bfd07ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS| CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fd3bfd089d0, tls=0x7fd3bfd08700, child_tidptr=0x7fd3bfd089d0) = 3 2637
580 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7fd3bed07000
581 mprotect(0x7fd3bed07000, 4096, PROT_NONE) = 0
582 clone(child_stack=0x7fd3bf506ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS| CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7fd3bf5079d0, tls=0x7fd3bf507700, child_tidptr=0x7fd3bf5079d0) = - 1 EAGAIN (Resource temporarily unavailable)
583 futex(0x253b420, FUTEX_WAKE_PRIVATE, 1) = 1
584 futex(0x253b3f4, FUTEX_CMP_REQUEUE_PRIVATE, 1, 2147483647, 0x253b420, 4) = 2
585 futex(0x253b420, FUTEX_WAKE_PRIVATE, 1) = 1
586 futex(0x7fd3c2d0e9d0, FUTEX_WAIT, 32631, NULL) = 0
587 munmap(0x7fd3bed07000, 8392704) = 0
588 munmap(0x7fd3c250e000, 8392704) = 0
589 munmap(0x7fd3c1d0d000, 8392704) = 0
590 munmap(0x7fd3c150c000, 8392704) = 0
591 write(2, "\33[1;32m[AVFilterGraph # 0x253b0a"..., 39[AVFilterGraph # 0x253b0a0] ) = 39
592 write(2, "\33[1;31mError initializing thread"..., 41Error initializing threading.
593 ) = 41
594 write(2, "\33[1;32m[AVFilterGraph # 0x253b0a"..., 39[AVFilterGraph # 0x253b0a0] ) = 39
595 write(2, "\33[1;31mError creating filter 'an"..., 41Error creating filter 'anull'
596 ) = 41
597 write(2, "\33[4;31mError opening filters!\n\33["..., 34Error opening filters!
598 ) = 34
strace -e non root:
strace -e open ffmpeg -i 1.mov 5.avi
open("/etc/ld.so.cache", O_RDONLY) = 3
open("/lib64/libasound.so.2", O_RDONLY) = 3
open("/usr/lib64/libSDL-1.2.so.0", O_RDONLY) = 3
open("/lib64/libpthread.so.0", O_RDONLY) = 3
open("/usr/lib64/libx264.so.142", O_RDONLY) = 3
open("/usr/lib64/libvpx.so.0", O_RDONLY) = 3
open("/usr/local/lib/libvorbisenc.so.2", O_RDONLY) = 3
open("/usr/local/lib/libvorbis.so.0", O_RDONLY) = 3
open("/usr/local/lib/libmp3lame.so.0", O_RDONLY) = 3
open("/lib64/libm.so.6", O_RDONLY) = 3
open("/lib64/libbz2.so.1", O_RDONLY) = 3
open("/usr/local/lib/libz.so.1", O_RDONLY) = 3
open("/lib64/librt.so.1", O_RDONLY) = 3
open("/lib64/libc.so.6", O_RDONLY) = 3
open("/lib64/libdl.so.2", O_RDONLY) = 3
open("/usr/local/lib/libogg.so.0", O_RDONLY) = 3
ffmpeg version N-65949-g0ddb051 Copyright (c) 2000-2014 the FFmpeg developers
built on Aug 28 2014 11:39:47 with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-4)
configuration: --enable-gpl --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264
libavutil 54. 7.100 / 54. 7.100
libavcodec 56. 0.101 / 56. 0.101
libavformat 56. 3.100 / 56. 3.100
libavdevice 56. 0.100 / 56. 0.100
libavfilter 5. 0.103 / 5. 0.103
libswscale 3. 0.100 / 3. 0.100
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 0.100 / 53. 0.100
open("1.mov", O_RDONLY|O_CLOEXEC) = 3
open("/etc/localtime", O_RDONLY) = 4
open("/proc/meminfo", O_RDONLY|O_CLOEXEC) = 4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mov':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2014-06-05 04:40:27
model : iPhone 5
model-eng : iPhone 5
encoder : 7.1.1
encoder-eng : 7.1.1
date : 2014-06-04T21:40:27-0700
date-eng : 2014-06-04T21:40:27-0700
make : Apple
make-eng : Apple
Duration: 00:00:07.15, start: 0.000023, bitrate: 781 kb/s
Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m), 480x360, 710 kb/s, 29.09 fps, 600 tbr, 600 tbn, 1200 tbc (default)
Metadata:
rotate : 90
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
encoder : H.264
Side data:
displaymatrix: rotation of -90.00 degrees
Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 63 kb/s (default)
Metadata:
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
open("5.avi", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 4
[AVFilterGraph # 0x26ff4a0] Error initializing threading.
[AVFilterGraph # 0x26ff4a0] Error creating filter 'anull'
Error opening filters!
strace root:
579 clone(child_stack=0x7f4eb7c34ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS| CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f4eb7c359d0, tls=0x7f4eb7c35700, child_tidptr=0x7f4eb7c359d0) = 3 383
580 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f4eb6c34000
581 mprotect(0x7f4eb6c34000, 4096, PROT_NONE) = 0
582 clone(child_stack=0x7f4eb7433ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS| CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f4eb74349d0, tls=0x7f4eb7434700, child_tidptr=0x7f4eb74349d0) = 3 384
583 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f4eb6433000
584 mprotect(0x7f4eb6433000, 4096, PROT_NONE) = 0
585 clone(child_stack=0x7f4eb6c32ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS| CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f4eb6c339d0, tls=0x7f4eb6c33700, child_tidptr=0x7f4eb6c339d0) = 3 385
586 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f4eb5c32000
587 mprotect(0x7f4eb5c32000, 4096, PROT_NONE) = 0
588 clone(child_stack=0x7f4eb6431ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS| CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f4eb64329d0, tls=0x7f4eb6432700, child_tidptr=0x7f4eb64329d0) = 3 386
589 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f4eb5431000
590 mprotect(0x7f4eb5431000, 4096, PROT_NONE) = 0
591 clone(child_stack=0x7f4eb5c30ff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS| CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f4eb5c319d0, tls=0x7f4eb5c31700, child_tidptr=0x7f4eb5c319d0) = 3 387
592 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f4eb4c30000
593 mprotect(0x7f4eb4c30000, 4096, PROT_NONE) = 0
594 clone(child_stack=0x7f4eb542fff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS| CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f4eb54309d0, tls=0x7f4eb5430700, child_tidptr=0x7f4eb54309d0) = 3 388
595 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f4eb442f000
596 mprotect(0x7f4eb442f000, 4096, PROT_NONE) = 0
597 clone(child_stack=0x7f4eb4c2eff0, flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND|CLONE_THREAD|CLONE_SYSVSEM|CLONE_SETTLS| CLONE_PARENT_SETTID|CLONE_CHILD_CLEARTID, parent_tidptr=0x7f4eb4c2f9d0, tls=0x7f4eb4c2f700, child_tidptr=0x7f4eb4c2f9d0) = 3 389
598 mmap(NULL, 8392704, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS|MAP_STACK, -1, 0) = 0x7f4eb3c2e000
strace -e root:
strace -e open ffmpeg -i 1.mov 5.avi
open("/etc/ld.so.cache", O_RDONLY) = 3
open("/lib64/libasound.so.2", O_RDONLY) = 3
open("/usr/lib64/libSDL-1.2.so.0", O_RDONLY) = 3
open("/lib64/libpthread.so.0", O_RDONLY) = 3
open("/usr/lib64/libx264.so.142", O_RDONLY) = 3
open("/usr/lib64/libvpx.so.0", O_RDONLY) = 3
open("/usr/local/lib/libvorbisenc.so.2", O_RDONLY) = 3
open("/usr/local/lib/libvorbis.so.0", O_RDONLY) = 3
open("/usr/local/lib/libmp3lame.so.0", O_RDONLY) = 3
open("/lib64/libm.so.6", O_RDONLY) = 3
open("/lib64/libbz2.so.1", O_RDONLY) = 3
open("/usr/local/lib/libz.so.1", O_RDONLY) = 3
open("/lib64/librt.so.1", O_RDONLY) = 3
open("/lib64/libc.so.6", O_RDONLY) = 3
open("/lib64/libdl.so.2", O_RDONLY) = 3
open("/usr/local/lib/libogg.so.0", O_RDONLY) = 3
ffmpeg version N-65957-gc6a3b00 Copyright (c) 2000-2014 the FFmpeg developers
built on Aug 28 2014 13:37:32 with gcc 4.4.7 (GCC) 20120313 (Red Hat 4.4.7-4)
configuration: --enable-gpl --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264 --prefix=/usr --libdir=/usr/lib64 --enable-gpl --enable-libmp3lame --enable-libvorbis --enable-libvpx --enable-libx264
libavutil 54. 7.100 / 54. 7.100
libavcodec 56. 0.101 / 56. 0.101
libavformat 56. 3.100 / 56. 3.100
libavdevice 56. 0.100 / 56. 0.100
libavfilter 5. 0.103 / 5. 0.103
libswscale 3. 0.100 / 3. 0.100
libswresample 1. 1.100 / 1. 1.100
libpostproc 53. 0.100 / 53. 0.100
open("1.mov", O_RDONLY|O_CLOEXEC) = 3
open("/etc/localtime", O_RDONLY) = 4
open("/proc/meminfo", O_RDONLY|O_CLOEXEC) = 4
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '1.mov':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
creation_time : 2014-06-05 04:40:27
model : iPhone 5
model-eng : iPhone 5
encoder : 7.1.1
encoder-eng : 7.1.1
date : 2014-06-04T21:40:27-0700
date-eng : 2014-06-04T21:40:27-0700
make : Apple
make-eng : Apple
Duration: 00:00:07.15, start: 0.000023, bitrate: 781 kb/s
Stream #0:0(und): Video: h264 (Baseline) (avc1 / 0x31637661), yuv420p(tv, smpte170m), 480x360, 710 kb/s, 29.09 fps, 600 tbr, 600 tbn, 1200 tbc (default)
Metadata:
rotate : 90
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
encoder : H.264
Side data:
displaymatrix: rotation of -90.00 degrees
Stream #0:1(und): Audio: aac (mp4a / 0x6134706D), 44100 Hz, mono, fltp, 63 kb/s (default)
Metadata:
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
File '5.avi' already exists. Overwrite ? [y/N] y
open("5.avi", O_WRONLY|O_CREAT|O_TRUNC|O_CLOEXEC, 0666) = 4
Output #0, avi, to '5.avi':
Metadata:
major_brand : qt
minor_version : 0
compatible_brands: qt
make-eng : Apple
model : iPhone 5
model-eng : iPhone 5
make : Apple
ISFT : Lavf56.3.100
ICRD : 2014-06-04T21:40:27-0700
date-eng : 2014-06-04T21:40:27-0700
Stream #0:0(und): Video: mpeg4 (FMP4 / 0x34504D46), yuv420p, 480x360, q=2-31, 200 kb/s, 29.09 fps, 29.09 tbn, 29.09 tbc (default)
Metadata:
rotate : 90
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
encoder : Lavc56.0.101 mpeg4
Stream #0:1(und): Audio: mp3 (libmp3lame) (U[0][0][0] / 0x0055), 44100 Hz, mono, fltp (default)
Metadata:
creation_time : 2014-06-05 04:40:27
handler_name : Core Media Data Handler
encoder : Lavc56.0.101 libmp3lame
Stream mapping:
Stream #0:0 -> #0:0 (h264 (native) -> mpeg4 (native))
Stream #0:1 -> #0:1 (aac (native) -> mp3 (libmp3lame))
Press [q] to stop, [?] for help
frame= 204 fps=0.0 q=27.2 Lsize= 348kB time=00:00:07.15 bitrate= 398.4kbits/s dup=0 drop=4
video:271kB audio:56kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 6.560985%
The strace output shows that the non-root run fails its second call to clone with EAGAIN. According to the manual pages clone(2) and fork(2), this means the system has reached a limit on the number of threads the user may create.
The likely culprit is the per-process resource limit. Run ulimit -u to see what the current process's limit is, then try raising it and trying again.

Unknown Memory leak in android

adb shell dumpsys meminfo of my package shows the following and my native allocated size increases and finally causing mobile restart. Is that any memory ? How can i fix that??
native dalvik other total limit bitmap nativeBmp
size: 445456 5955 N/A 451411 32768 N/A N/A
allocated: 445024 3726 N/A 448750 N/A 10948 1912
free: 43 2229 N/A 2272 N/A N/A N/A
(Pss): 132631 870 300292 433793 N/A N/A N/A
(shared dirty): 2532 1656 5552 9740 N/A N/A N/A
(priv dirty): 132396 708 298960 432064 N/A N/A N/A
Objects
Views: 0 ViewRoots: 0
AppContexts: 0 Activities: 0
Assets: 6 AssetManagers: 6
Local Binders: 5 Proxy Binders: 14
Death Recipients: 1
OpenSSL Sockets: 0
SQL
heap: 0 MEMORY_USED: 0
PAGECACHE_OVERFLOW: 0 MALLOC_SIZE: 50
Asset Allocations
zip:/data/app/com.outlook.screens-2.apk:/resources.arsc: 25K
zip:/data/app/com.outlook.screens-2.apk:/assets/font/RobotoCondensedRegular.
ttf: 156K
zip:/data/app/com.outlook.screens-2.apk:/assets/font/RobotoCondensedBold.ttf
: 158K
zip:/data/app/com.outlook.screens-2.apk:/assets/font/RobotoCondensedLight.tt
f: 157K
Uptime: 190161845 Realtime now=432619753

FFMPEG av_interleaved_write_frame(): Operation not permitted

Ok I receiving a 'av_interleaved_write_frame(): Operation not permitted' error while trying to encode an MOV file. Firstly I need to outline the conditions behind it.
I am encoding 12 different files of different resolution sizes and format types via a PHP script that runs on cron. Basically it grabs a 250mb HD MOV file and encodes it in 4 different frame sizes as MOV, MP4 and WMV file types.
Now the script takes over 10mins to run and encode each of the files for the 250mb input file. I am outputting the processing times and as soon as the time on the script hits 10mins FFMPEG crashes and returns "av_interleaved_write_frame(): Operation not permitted" for the current file being encoded and all other remaining files yet to be encoded.
If the input videos is 150MB the total time the script runs for is under 10mins so it encodes all of the videos fine. Additionally if I run the FFMPEG command on the individual file that it fails on for the 250mb file it encodes the file with no issues.
From doing to research on the error "av_interleaved_write_frame()" it seems it is related to timestamps of what I understand to be of the input file. But in saying that it doesn't seem to be the case in my instance because I can encode the file with no problem if I do it individually.
example ffmpeg command
ffmpeg -i GVowbt3vsrXL.mov -s 1920x1080 -sameq -vf "unsharp" -y GVowbt3vsrXL_4.wmv
Error output on the failed file at 10mins. Remember there is no issue with the command if I run it by itself it is only when the script hits 10mins.
'output' =>
array (
0 => 'FFmpeg version SVN-r24545, Copyright (c) 2000-2010 the FFmpeg developers',
1 => ' built on Aug 20 2010 23:32:02 with gcc 4.1.2 20080704 (Red Hat 4.1.2-48)',
2 => ' configuration: --enable-shared --enable-gpl --enable-pthreads --enable-nonfree --cpu=opteron --extra-cflags=\'-O3 -march=opteron -mtune=opteron\' --enable-libmp3lame --enable-libx264 --enable-libxvid --enable-avfilter --enable-filter=movie --enable-avfilter-lavf --enable-swscale',
3 => ' libavutil 50.23. 0 / 50.23. 0',
4 => ' libavcore 0. 1. 0 / 0. 1. 0',
5 => ' libavcodec 52.84. 1 / 52.84. 1',
6 => ' libavformat 52.77. 0 / 52.77. 0',
7 => ' libavdevice 52. 2. 0 / 52. 2. 0',
8 => ' libavfilter 1.26. 1 / 1.26. 1',
9 => ' libswscale 0.11. 0 / 0.11. 0',
10 => 'Input #0, mov,mp4,m4a,3gp,3g2,mj2, from \'/home/hdfootage/public_html/process/VideoEncode/_tmpfiles/GVowbt3vsrXL/GVowbt3vsrXL.mov\':',
11 => ' Metadata:',
12 => ' major_brand : qt',
13 => ' minor_version : 537199360',
14 => ' compatible_brands: qt',
15 => ' Duration: 00:00:20.00, start: 0.000000, bitrate: 110802 kb/s',
16 => ' Stream #0.0(eng): Video: mjpeg, yuvj422p, 1920x1080 [PAR 72:72 DAR 16:9], 109386 kb/s, 25 fps, 25 tbr, 25 tbn, 25 tbc',
17 => ' Stream #0.1(eng): Audio: pcm_s16be, 44100 Hz, 2 channels, s16, 1411 kb/s',
18 => '[buffer # 0xdcd0e0] w:1920 h:1080 pixfmt:yuvj422p',
19 => '[unsharp # 0xe00280] auto-inserting filter \'auto-inserted scaler 0\' between the filter \'src\' and the filter \'Filter 0 unsharp\'',
20 => '[scale # 0xe005b0] w:1920 h:1080 fmt:yuvj422p -> w:1920 h:1080 fmt:yuv420p flags:0xa0000004',
21 => '[unsharp # 0xe00280] effect:sharpen type:luma msize_x:5 msize_y:5 amount:1.00',
22 => '[unsharp # 0xe00280] effect:none type:chroma msize_x:0 msize_y:0 amount:0.00',
23 => 'Output #0, asf, to \'/home/hdfootage/public_html/process/VideoEncode/_tmpfiles/GVowbt3vsrXL/GVowbt3vsrXL_4.wmv\':',
24 => ' Metadata:',
25 => ' WM/EncodingSettings: Lavf52.77.0',
26 => ' Stream #0.0(eng): Video: msmpeg4, yuv420p, 1920x1080 [PAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 1k tbn, 25 tbc',
27 => ' Stream #0.1(eng): Audio: libmp3lame, 44100 Hz, 2 channels, s16, 64 kb/s',
28 => 'Stream mapping:',
29 => ' Stream #0.0 -> #0.0',
30 => ' Stream #0.1 -> #0.1',
31 => 'Press [q] to stop encoding',
32 => '[msmpeg4 # 0xdccb50] warning, clipping 1 dct coefficients to -127..127',
Then it errors
frame= 75 fps= 5 q=1.0 size= 12704kB time=2.90 bitrate=3588 6.0kbits av_interleaved_write_frame(): Operation not permitted',
)
Has any anybody encountered this sort of problem before? It seems to be something to do with the timestamps but only because the script is running for a period longer then 10mins. It maybe related to PHP/Apache config but I don't know if it is FFMPEG or if it is server config I need to be looking at.
One trivially solvable cause for this problem (if you found this question while searching for this problem, as I did) is when the partition you are trying to write your file to is full. Make sure you have enough available space to write your file to. On linux/unix this is as easy as running
$ df -h
To solve, just free up enough space by moving files to another partition, deleting unwanted files, and emptying your trash.
this patch (linked from ffmpeg issue 807's page) solved the problem for me reencoding video and copying audio out of a live captured flv file to an avi file:
https://roundup.ffmpeg.org/file1098/utils.c.patch
I encounter this issue when I was trying to mux 1920x1080 video in mpeg transportstream using custom AVIOContext like this:
mAVIOBufferSize = 1024 * 10;
mAVIOBuffer = static_cast<unsigned char *>(av_malloc(mAVIOBufferSize));
mAVIOContext = avio_alloc_context(
mAVIOBuffer,
mAVIOBufferSize,
1,
this,
NULL,
write_packet,
NULL
);
mOutputContext->pb = mAVIOContext;
The problem was that my mAVIOBufferSize was too small for some of the the data av_interleaved_write_frame function wanted to pass to write_packet function.
I solved this by increasing the mAVIOBufferSize
mAVIOBufferSize = 1024 * 1024;
And the error disappeared.

Resources