Fix TeamNewPipe/NewPipe#2615

[Stypox]

This is the fix, but there some things that could be improved. Now the key URL_ENCODED_FMT_STREAM_MAP (line 623) is unused and has been replaced by ADAPTIVE_FMTS in the only place it was used: getVideoStreams(). I don't know if this is ok.
Also, I have added a field to the YoutubeStreamExtractor, playerResponse, to avoid extracting it every time. But I didn't use the class field in all places possible since I do not have time just now.
Also, now stream urls seem to have no signature ("s" or "sig") anymore, I tried with a video and every provided url worked. Still I don't know if this is ok.
Fixes TeamNewPipe/NewPipe#2615

This pr is probably ready, here is a debug apk: app-debug.zip
Please test whether all the video & audio formats work (I don't have much time to test now)

[omarroth]

url_encoded_fmt_stream_map provides formats with video+audio, while adaptiveFormats will only provide either audio or video. The equivalent for url_encoded_fmt_stream_map is formats in player_response. I expect NewPipe already has a mechanism for combining these, but it's something to keep in mind.

stream urls seem to have no signature

The sig, sp, and are included in a new field cipher (as mentioned in the comment above), along with the URL (if the signature is encrypted). I would recommend looking at this code for reference.

[omarroth]

A format with an encrypted signature will look like:

{
  "itag": 18,
  "mimeType": "video/mp4; codecs=\"avc1.42001E, mp4a.40.2\"",
  "bitrate": 190743,
  "width": 480,
  "height": 360,
  "lastModified": "1544771185080097",
  "contentLength": "7205319",
  "quality": "medium",
  "qualityLabel": "360p",
  "projectionType": "RECTANGULAR",
  "averageBitrate": 190708,
  "audioQuality": "AUDIO_QUALITY_LOW",
  "approxDurationMs": "302254",
  "audioSampleRate": "44100",
  "audioChannels": 2,
  "cipher": "s=yg%3DgbkvHeAj%3DBSI4qxNP853HXnP951uEILIUOrgo_Y5wk3AEiAXz4R5ABwKqpzTJx9DoJ0EN0ycyk4Gq_NSRYqkzSx8PKAhIgRww2IxgLALl&sp=sig&url=https%3A%2F%2Fr3---sn-pobpb-poql.googlevideo.com%2Fvideoplayback%3Fexpire%3D1568244103%26ei%3DJi15XaihPNntigTghIGYCw%26ip%3D137.112.229.227%26id%3Do-AK2uenby5A6kGWn98PjZ2RqOdZwvslnWvgwcHV1on5Rn%26itag%3D18%26source%3Dyoutube%26requiressl%3Dyes%26mm%3D31%252C29%26mn%3Dsn-pobpb-poql%252Csn-vgqskn7l%26ms%3Dau%252Crdu%26mv%3Dm%26mvi%3D2%26pl%3D16%26initcwndbps%3D2403750%26mime%3Dvideo%252Fmp4%26gir%3Dyes%26clen%3D7205319%26ratebypass%3Dyes%26dur%3D302.254%26lmt%3D1544771185080097%26mt%3D1568222392%26fvip%3D5%26beids%3D9466587%26c%3DWEB%26txp%3D5531432%26sparams%3Dexpire%252Cei%252Cip%252Cid%252Citag%252Csource%252Crequiressl%252Cmime%252Cgir%252Cclen%252Cratebypass%252Cdur%252Clmt%26lsparams%3Dmm%252Cmn%252Cms%252Cmv%252Cmvi%252Cpl%252Cinitcwndbps%26lsig%3DAHylml4wRQIgSkQ-E1LSZbLpaMHHKh_QAVBmZrUgGlRlrby82czu1J4CIQDh4b5yoIZu4rt_6VKSGa8GiqiYhKQFgpLeKyE_X6gm1g%253D%253D"
}

[Stypox]

Using the "cipher" content you provided I implemented the signature decoding. This is the url I generate, it works. :-D Please check it anyway @omarroth , thank you :-)
https://r3---sn-pobpb-poql.googlevideo.com/videoplayback?expire=1568244103&ei=Ji15XaihPNntigTghIGYCw&ip=137.112.229.227&id=o-AK2uenby5A6kGWn98PjZ2RqOdZwvslnWvgwcHV1on5Rn&itag=18&source=youtube&requiressl=yes&mm=31%2C29&mn=sn-pobpb-poql%2Csn-vgqskn7l&ms=au%2Crdu&mv=m&mvi=2&pl=16&initcwndbps=2403750&mime=video%2Fmp4&gir=yes&clen=7205319&ratebypass=yes&dur=302.254&lmt=1544771185080097&mt=1568222392&fvip=5&beids=9466587&c=WEB&txp=5531432&sparams=expire%2Cei%2Cip%2Cid%2Citag%2Csource%2Crequiressl%2Cmime%2Cgir%2Cclen%2Cratebypass%2Cdur%2Clmt&lsparams=mm%2Cmn%2Cms%2Cmv%2Cmvi%2Cpl%2Cinitcwndbps&lsig=AHylml4wRQIgSkQ-E1LSZbLpaMHHKh_QAVBmZrUgGlRlrby82czu1J4CIQDh4b5yoIZu4rt_6VKSGa8GiqiYhKQFgpLeKyE_X6gm1g%3D%3D&sig=ALgxI2wwRgIhAKP8xSzkqYRSN_qG4kycy0NE0JoD9xJTzpqKwBl5R4zXAiEA3kw5Y_ogrOUILIEu159PnXH358PNxq4ISByjAeHvkbg=

[TobiGr]

@Stypox and @omarroth thank you both for the effort ❤️

@Stypox It seems like only 480p videos work with my Android 4 emulator. I'll test it on my phone soon to verify and try to gather some more info.

[luantak]

@Stypox Tested the apk you provided, seems to work fine regarding video playback, but i have the issue that some videos play sound at resolution change on mute for about 0.25 Seconds then get muted again.

Edit: Livestreams don't work.

Device: Oneplus 6 w/ Android 9 (Lineage OS 16)

[TobiGr]

All video resolutions work fine on my phone.

[TobiGr]

@lu4p Can you give us a video url?
If I cannot reproduce, I'd suggest to merge and release a new version.

I tested some more videos and did not encounter any problems you mentioned.

[luantak]

@lu4p Can you give us a video url?
If I cannot reproduce, I'd suggest to merge and release a new version.

Happens on every video I tried. Try for example: https://www.youtube.com/watch?v=-Irhs4WkDE0

0. Detach Headphones
1. Set media volume to 0
2. Open Newpipe
3. Play a video in 360p
4. Switch to 1080p
   Now you can hear the video sound briefly (0.25 seconds) on speaker.

[luantak]

Try for example: https://www.youtube.com/watch?v=-Irhs4WkDE0

[TobiGr]

Cannot reproduce. Nevertheless, this has nothing to do with this fix. Please open a separate issue in the front-end repository.

[TobiGr]

Some tests are still failing, but I think we can take care of them later this week. Thanks again.
@theScrabi Please decide where to merge this fix. IMO, we can put this PR on top of the other changes which are already merged since the last release.

[kubo6472]

Just to let ya know. "Hintergrund-Player" works

[theScrabi]

All cool but dev was the wrong branch to merge into.
Please merge hotfix always into master.

[TobiGr]

@theScrabi The dev branch is stable as we only added fixes to it since the last release. I thought, it would be good to release these, too.

[TobiGr]

What yellow icon thing????

[ghost]

A yellow app icon, it was working for awhile, but it keeps breaking...I'll send you a screenshot if I can pull it down again...

On Thursday, September 12, 2019, Tobias Groza ***@***.***> wrote: What yellow icon thing???? — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.<

https://ci6.googleusercontent.com/proxy/AdYxYvM-LQ6soLlaTUzers3-EbDDgePuxV6xG3tOXOOZN33d1MNml1FZ3t1iN-xryZRFD12SGLYMTF_nLkqn5tZjkvkcVZNkIJhz3FnK7Muewk5um2D-lU40uFGmxz91P4hNW2XCMREyODI4dKkGKA1c1v5j85nlYPCQHTYMnGVUL6FmZzoN0ElW3RlGmzO0rkOD99SQjn_ro0FoBXwfD4o0EAlCXCshNi9G8FEmlQ=s0-d-e1-ft#https://github.com/notifications/beacon/AMHMGMMIXWVBPYA52MMRUHLQJFJO5A5CNFSM4IVWTF62YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD6P2INA.gif

…

-- Android strong, be all you can be, end transmission.

[ghost]

[ghost]

The yellow icon at the bottom right...

[TobiGr]

That's a debug tool which helps us to find memory leaks and other stuff. it is not included into the normal/release version. When you click it, it analyses the RAM used by NewPipe and therefore freeze the app until it has done its job.

[ghost]

I'm not a computer programmer, just a user, what's happening is that I have to go into the zip file in my files app and reinstall the thing everytime I go back into the New Pipe in order for it to work... I also have to reset all the settings in the New Pipe each time.

[TobiGr]

Just uninstall the app and install the release from here https://github.com/TeamNewPipe/NewPipe/releases/tag/v0.17.2. In case you installed NewPipe via F-Droid, you need to wait a day or two until F-Droid has the update.

[ghost]

Okay, I'll try that tomorrow, I did install it from F-droid, 17.1 version, I understand it might take a couple of days, thanks...