How to modify the 3gp streaming filter to read Only rtp data?

How to modify the 3gp streaming filter to read Only rtp data?
================================================
( 3gp streaming stack it will receives the rtsp and rtp data)

Solution:Quick and dirty way to do this:
===============================
Use Some SDP file to create the dummy audio and video output pin. ( We can create it with rtsp:// URL itself);
Hard code the media type, File duration (instead of the file duration configure duration as much as we need to receive data for RTP),config data in a filter .

RTP video and audio Timestamp Interval between RTP packets

RTP video and audio Timestamp Interval between RTP packets:
------------------------------------------------------------
For Audio, the timestamp is incremented by the packetization interval times the sampling rate. For example, for audio packets containing 20 ms of audio sampled at 8,000 Hz, the timestamp for each block of audio increases by ((8000 / 1000) * 20) i.e. by 160.
For a 30 f/s video, timestamps would increase by (sample rate / 30) i.e. by 3,000 for each frame. ( SampleRate is 90000 Hz for video).If a frame is transmitted as several RTP packets, these packets would all bear the same timestamp.
ts_unit is equal to the above ...

AMR Muxer for RTP

AMR Muxer for RTP:
--------------------
1.Input : AMR File
2.Output : AMR RTP Packet Data ( which contains 500 ms of data..
25 Frames * 20 ms)
Read 25 Frames and make it as like RTP amr data as follows:
i) F0,25 frames header and AMR Data
ii)


I have to test the RTP Muxer completely;

RTP Muxer is having problem;
RTP Demuxer will do the following:
1.F0 BC BC BC ... 3C
2.Number of BC's indicate the number of frames + 1 is number of frames in a RTP packet;
RTP muxer will have 24 BCs and 1 3C

About this BC value;
we can generate this value according the selected type:
3C refers 12.2 kbps
0111
1 0111 1 00

F FT Q 00
1 0111 1 00
F indicates the continuation of another Speech frame;
if F is 0 means that is the last speech frame;
After BC's, 3C is the last speech frame;
the Upper most bit is zero , so it is the last frame;
1 indicates continuation;

In case of 3C just check it...
0 0111 1 00


RTP Stack is sending the data correctly;
_ Working welland I have received the data properly

How can we sync audio and video in RTP

How can we sync audio and video in RTP:
-----------------------------------------
1.Receive RTP video and RTP audio in a separate filter
we will get RTCP packets for N number of RTP packets constantly;
For RTP video, we will get RTCP NTP timestamp and RTP timestamp; and at the same time, we will get the RTP packet and
Extract it Timestamp;
NTP - is network time; it is used as a reference clock;

NTP is common to audio and video; we are adjusting avideo based on NTP and adjusting audio based on NTP; So it makes them be in sync;

Do the calculation as follows:

Video Timestamp = (RTP Timestamp of RTCP video packet / NTP timestamp of RTCP video packet) * RTP video Packet's RTP timestamp;
Audio Timestamp = (RTP Timestamp of RTCP Audio packet / NTP timestamp of RTCP Audio packet) * RTP audio Packet's RTP timestamp;

RTCP is sent +1 port of the RTP;
RTP Video Port number = 5000;
RTCP video Port number = RTP Video Port number + 1; ( 5001)
RTP audio Port number = 6000;
RTCP audio Port number = RTP audio Port number + 1; ( 6001 )

ffmpeg rtp and rtcp audio video problem

if I am sending audio and vide thru ffmpeg, I got flickering in the video;
the reason is ffmpeg video 4999 audio 5000
But I am not getting proper audio at client side; I got some glitches .
Reason :
---------------
1.we have to choose the even number of rtp port;rtcp packets will be sent by the ffmpeg ( rtp port + 1) = rtcp port number;
2. rtp video is sent in 4999 and ffmpeg will send the rtcp info at 5000
3.At the same time ffmpeg will sends audio at RTP port 5000 and RTCP at 5001;
4.So FFMPEG is sending the RTP and RTCP packets in a same Port that is 5000;

Audio and Video Port Problem in Audio Testing

RTP AMRTesting :
=================
1.I combined the multiple RTP packets in to a Single packet and passed it to the AMR Paser application;
2.Next added the AMR file extension and AMR start codes and try to play it with QT Player;
Created AMR file is wroking with QT Player; It is working.

3.Pls kindly check the Market Bit of the RTP header format;

if we are sending the audio and video thru INVC transcoder,I printed the RTP packet Number as follows:
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 60, Buffer size = 1389, Marker = 0
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 61, Buffer size = 1389, Marker = 0
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 62, Buffer size = 1389, Marker = 0
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 6, Buffer size = 28, Marker = 128
RTP packet Number = 63, Buffer size = 1389, Marker = 0

ffmpeg sends amr audio to RTP.I got the following:
RTP packet Number = 3, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 3
RTP packet Number = 4, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 4
RTP packet Number = 5, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 5
RTP packet Number = 6, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 6
RTP packet Number = 7, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 7
RTP packet Number = 8, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 8
RTP packet Number = 9, Buffer size = 1389, Marker = 0 HSB = 0 , LSB= 9


while sending audio and video we have the problem;One more thing if we send audio and video, QuickTime player is able to render the audio.


So without the packet number we need to do parsing;
Every Audio Frame Beginning is marked by the Marker bit as Zero;otherwise it will be 1 or TRUE;
If I developed the amr parser for this scenario, It causes the Screech Sound in audio with particular interval.At client side, I received the RTP packets and Dumped it into a file also givesthe same problem;

Solution:
-----------------

Finally I got the Solution:
we are sending video data at 5000 Port We are sending audio data at 5001 Port ;So it causes the problem;
Instead of it If I changed the things as
video as 4500 and Audio as 5000 means there will be no problem So Every RTP amr packet is having header Info;
For Audio and video There must be some difference.So that we can avoid this situation;

How to stream AMR to RTP and Play it in QuickTime

Use AMR file as source to stream AMR to RTP. ffmpeg -re -i "d:\media\vel1.amr" -acodec libamr_nb -ar 8000 -ab 12.2kb -ac 1 -f rtp rtp://192.168.1.198:5000/

it plays the audio in Quicktime.
AMR.SDP File contents:
-----------------------
v=0
o=Mass 3123312 121232 IN IP4 192.168.1.198
s=Rtsp Session
c=IN IP4 192.168.1.198
t=0 0
a=range:npt=0-
m=audio 5000 RTP/AVP 97
a=rtpmap:97 AMR/8000
a=fmtp:97 octet-align=1

RTP audio filter problem in latency...

1.Yesterday we completed the Packet loss adjusting mechanism.
2.Today we improved the audio quality. By Putting IAmPushSource and set the latency as
GetLatency(){ *plLatency = 3500000; return S_OK;}
I have implemented the IAMPushSource on My live source Filter's Output Pin.
I faced 2 to 3 seconds latency problem;
So I modified the things as follows:
STDMETHODIMP CRTPAudioStream::GetPushSourceFlags(ULONG *pFlags)
{
*pFlags = AM_PUSHSOURCECAPS_PRIVATE_CLOCK;
//The filter time stamps the samples using a private clock.
//The clock is not available to the rest of the graph through IReferenceClock.
return S_OK;
}
Now everything works fine within the Filter, we implemented as follows:

class CRTPFilter: public CSourceStream, public IAMFilterMiscFlags
{
public: // IAMFilterMiscFlags override
ULONG STDMETHODCALLTYPE GetMiscFlags(void)
{ return AM_FILTER_MISC_FLAGS_IS_SOURCE; }
};

class CRTPAudioStream : public CSourceStream,public IAMPushSource
{
public:
//IAMPushSource
STDMETHODIMP GetMaxStreamOffset(REFERENCE_TIME *prtMaxOffset)
{return E_NOTIMPL;}
STDMETHODIMP GetPushSourceFlags(ULONG *pFlags)
{ *pFlags = AM_PUSHSOURCECAPS_PRIVATE_CLOCK;return S_OK;}
STDMETHODIMP GetStreamOffset(REFERENCE_TIME *prtOffset)
{return E_NOTIMPL;}
STDMETHODIMP SetMaxStreamOffset(REFERENCE_TIME rtMaxOffset)
{return E_NOTIMPL;}
STDMETHODIMP SetPushSourceFlags(ULONG Flags)
{return E_NOTIMPL;}
STDMETHODIMP SetStreamOffset(REFERENCE_TIME rtOffset)
{return E_NOTIMPL;}
STDMETHODIMP GetLatency(REFERENCE_TIME *prtLatency)
{ * prtLatency = 45000000; return S_OK;} //Set 450ms Latency for audio
};

RTP Audio Source Filter Problem without Glitches

Solution for Removing Audio Glitches in a RTP Audio Source Filter
-------------------------------------------------------------------
1.Test the Timestamp of the large files.
- it is also having some strange values like Start Time : 40000 and End Time :0
2.Set only start Time and be the stop time always 0.3.Dump the RTP Packet within the file.( No Problem with audio data)4.Implement the IAMPushSource interface.
I tested the large MP2 file in GraphEdit :
------------------------------------------------
If I inserted the dummy filter before the MPEG1 Audio Decoder filter, I got the following :
Start Time : 28114700 Stop Time : 23998200 Diff : 4116500Another time I got it as follows:
Start Time :31588108 Stop Time :28454648
Differnece :3133460

If I inserted the Dummy Filter after the MPEg1 Audio Decoder, I got the timestamp values as follows:
Grabber Sample size is : 18432 Start Time :0 Stop Time :0
Grabber Sample size is : 18432 Start Time :960000 Stop Time :0
Grabber Sample size is : 18432 Start Time : 1920000 Stop Time : 0
For the RTP Source Filter I got the following:
----------------------------------------------- 1.If I inserted the dummy filter before the MPEG1 Audio Decoder filter, I got the following :
Start Time Stop Time 2520000 2520000 2520000 2835000 2835000 3150000 2.If I inserted the Dummy Filter after the MPEG1 Audio Decoder, I got the timestamp values as follows:
Grabber Sample Size is : 18432 Start Time : 2520000 Stop Time : 0
Start Time : 2835000 Stop Time : 0 Start Time : 3150000 Stop Time : 0
Start Time : 3465000 Stop Time : 0 Start Time : 3780000 Stop Time : 0
Nothing above is works.
Solution:
-------------------

I am sending mp2 audio data with 48Hz frequency. At the RTP Audio Source Filter, I modified the frequency as 22.05 ( 22050 samples per second)..

Samples per second is the speed controller of a filter graph. it determines the speed of the filter.

48000(48 KHz) filter will run fast.
44100 (44.1KHz) filter will be some extent slow compare to the 48000.
22050( 22.05 KHz ) is very slow.
we are rendering the audio half of the transmitted audio's speed.

How can we extract the MPEG4 Frame Type from the RTP packet ?

How can we extract the MPEG4 Frame Type from the RTP packet (I,P or B):
------------------------------------------------------------------------
We are creating a software to parse MPEG4 data. Can anyone tell me how to extract the Frame Type (I,P,B) frame type from the whole RTP packet?
I mean I have RTP packet with MPEG4 payload in it, then how to extract the Frame type information from it?
Solution :
---------------
vop_coding_type is encoded in the first 2 bits after vop_start_code.
vop_start_code is 00 00 01 b6 (hex)
vop_coding_type means:
0 0 -I VOP
0 1 -P VOP
1 0 -B VOP
1 1 -S VOP

we are sending mpeg4 video thru rtp using ffmpeg.
Every I Frame( KeyFrame) is having VOS start code. 00 00 01 B0

Register the Protocol in Directshow

Register the Protocol for the Filter:
--------------------------------------
The objective is if we opened the protocol URL in a GraphEdit,we need to insert our filter and capture the data.
Example :
----------
I opened the RTP protocol as in GraphEdit as follows in Render URL option:
rtp://127.0.0.1:5000/
it must insert the Source Filter.
it is not having the separate Source Filter.
I developed the Source Filter. while rendering the RTP, I want to insert my filter.
what we need to do ?
---------------------
Develop the Registry setup for the RTP protocol with our source Filter
RTP.reg contents :
------------------
REGEDIT4
[HKEY_CLASSES_ROOT\rtp]@="RTP : Realtime Transfer Protocol""EditFlags"=hex:02,00,00,00"URL Protocol"="""Source Filter"="{fd501041-8ebe-11ce-8183-00aa00577da1}"
[HKEY_CLASSES_ROOT\rtp\DefaultIcon]@="c:\\Program Files\\Windows Media Player\\wmplayer.exe,0"
[HKEY_CLASSES_ROOT\rtp\QuickView]@="*"
[HKEY_CLASSES_ROOT\rtp\Shell]@=""
[HKEY_CLASSES_ROOT\rtp\Shell\open]"EditFlags"=hex:01,00,00,00
[HKEY_CLASSES_ROOT\rtp\Shell\open\command]@="C:\\Program Files\\Windows Media Player\\wmplayer.exe %1"
By double clicking this file we can link our source Filter with the RTP protocol.But In the source Filter we need to implement the IFileSourceFilter interface.
What I have done ? I implemented the IFileSourceFilter in a Ball Source Filter and need to insert this filter whenever the the RTP protocol is rendered. Then within Load and GetCurFile() fn we got the control in a source Filter.
If I opened the rtp stream in a Windows media player 9 ( wmplayer.exe) rtp://127.0.0.1:4000/,it is not working. But if I opened the rtp stream in windows media player 6( mplayer2.exe), it inserts the The ball filter and renders the data in a player.

mplayer2.exe
=============
How can we run the .reg file and modify the registry ?.
.------------------------------------------------------
start rtp.reg


Output of the RTP source Filter :
-------------------------------------
MEDIATYPE : MEDIATYPE_Video
Compression : DIVX
Width : 176
Height : 144

ffmpeg rate emulation for realtime streaming

we sent the MPEG4 Encoded data thru ffmpeg in a RTP stream.At receiver side, we are not getting the correct number of frames.( Sequence Number of the Frames highly differ).
Like After the RTP Packet (Sequence) number 1526, I got the number as 3600 or something.
ffmpeg sends the Entire data at with 5 minutes( which can be streamed around 30 mins).So there might be the chance for TCP/IP stack overflow. It may causes some packet loss.
(Real Time) if we are sending the video data as same as stream time,then there will be no packet loss. So we send the data to RTP stream using rate emulation ( -re ) command in ffmpeg.
Eventhough we are sending data with this one, we got the packet loss problem.
Check it with the Ethreal packet capture.
Capture RTP packets using Ethreal and RTP stack application at the same time.

RTP Source Filter or SDP Source Filter ?

Develop the Stories :

-----------------------

Way1 :

-------

1.Implement the IFileSourceFilter in RTP source filter

2.In IFileSourceFilter's Load() fn,Do the following:

i)if the VOS header is available, then Set this VOS header to MPEG2VIDEOINFO's dwSequenceHeader and Set the Media type MPEG2VIDEOINFO 's width and height and further informations

ii) Wait For voS or VOL header. if the VOS header is not available, then check for the VOL header. if the VOL header is available then Frame the VOS header based on VOL information and Hardcode the profile and level indication and construct the VOS header

3.Later convert it to Socket and Instead of RTP MPEG4 Source Filters, we may call it as SDP Source Filter. RTP Source Filter ? or SDP Source Filter ? There will be no RTP Source filter only SDP Source Filter? Combine more than one stream.

RTP Source Filter or SDP Source Filter ?

Develop the Stories :
-----------------------
Way1 :
-------
1.Implement the IFileSourceFilter in RTP source filter
2.In IFileSourceFilter's Load() fn,Do the following:

i)if the VOS header is available, then Set this VOS header to MPEG2VIDEOINFO's dwSequenceHeader and Set the Media type MPEG2VIDEOINFO 's width and height and further informations

ii) Wait For voS or VOL header. if the VOS header is not available, then check for the VOL header. if the VOL header is available then Frame the VOS header based on VOL information and Hardcode the profile and level indication and construct the VOS header

3.Later convert it to Socket and Instead of RTP MPEG4 Source Filters, we may call it as SDP Source Filter. RTP Source Filter ? or SDP Source Filter ? There will be no RTP Source filter only SDP Source Filter? Combine more than one stream.

How to parse MPEG4 data over the RTP stream and identify the header with SDP file ?

analysing the RTP received packets from middle of the sending and test the SDP with QuickTime and VLC player.
Qt and VLC plays the RTP stream thru SDP file .So There might be two chance for it.

1.without SDP file players can't receive RTP stream data.
Reason :
If I opened the RTP stream in a VLC player, it doesnt receive RTP stream data and gives unable to open error.

2. So based on this SDP file and we have to search for the VOL_start_code. For Each frame VOP is available. Successive VOPs can have I Frame or P Frame. for computer video, VOL_Header must have an information about Width and height

So combine the VOP information and SDP file information, it creates the header for MPEG4 Encoded data. But VOP start Code is not like that... 0000 01 B6 (vop_start_code ) it is available in more than one RTP packet consecutively.These packets are having less frame size.

VOL header is having more video Info . So it is having the header info. So we have to search for the Next VOL header.
if we started from middle of the RTP stream.

How to Receive RTP Stream in the middle of The Transmission ?

Currently we must receive RTP Stream from Start Onwards...
Then only it will works correctly.otherwise it may not work out.So Do this Excercise :
1.Receive the RTP Stream from Middle of Transmission and dump it into a file.
2.Pass this dump file to the ffmpeg and decode it by specifying the Profile-level-id and specify the format as m4v and can see if we can specify theES.
ffmpeg also decodes the .m4v file well.


Previously I thought of it as SDP file can be played only with QuickTime.QuickTime is having some kernel based capture driver to receive data from socket.Actually QuickTime processes are running in background.But now it is not like that.But Later I played the SDP file with VLC player. VLC player doesnt have an exe file running in background.So we can play the MPEG4 video elementary stream from the middle of the sTream.Further Stream Infos are not needed.

Filter needed for Distance Learning System

Filters Needed For Distance Learning System
:-------------------------------------------------
Based on RTP:
-----------------
1.RTP Video Sender Filter
2.RTP Video Receiver Filter
3.RTP Audio Sender Filter
4.RTP Audio Receiver Filter
5.MPEG4 Video Encoder Filter
6.MPEG4 Video Decoder Filter
7.MPEG4 Audio Encoder Filter
8.MPEG4 Audio Decoder Filter
9.Audio and Video Sync Filter


Based On RTSP:
------------------
1.RTSP Sender Filter
2.RTSP Receiver Filter
3.MPEG4 Multiplexer Filter
4.MPEG4 Demultiplexer Filter
5.MPEG4 Video Encoder Filter
6.MPEG4 Video Decoder Filter
7.MPEG4 Audio Encoder Filter
8.MPEG4 Audio Decoder Filter

Additional Features :
---------------------
1.MPEG4 Writer Filter for writing it into an MP4 File

How can we develop the Source Filter for RTP Receiver and set its media type ?...

How can we develop the Source Filter for RTP Receiver ?...
We can do one thing.

1.Develop the Source filter with Dynamic Output Pin,That means the output pin's buffer size may vary.

2.RTP receiver Filter must send the output data to its outptut pin based on the Frame. if the Frame is wrapped over more than one RTP packet, we have to send it over the network. For Doing this one develop the Separate class which will separate the RTP header and assemlble them as frame.
3.Look on MWS source Filter for Dynamic Output Pin.
GetStreamCount()
for(int i = 0; i < GetStreamCount(); i++)
{
StreamInfo stInfo = GetStreamInfo(i);
if(stInfo->id == VIDEO) { //Video is available in a Stream
stInfo->GetVideoFormat();
}
else if(stInfo->id == AUDIO)
{ //Audio is available in this stream stInfo->GetAudioFormat(); }
stInfo->GetExtraInfo() // Extra Header...
}
1.GetStreamInfo()
2.TrackId

VideoInfoHeader also must be set at the time of creating the Output Pin.

How to decode the RTP MPEG4 dump( received bits) ?

How to decode the RTP MPEG4 dump :
-----------------------------------------
I added the Network receiver filter and dump the received data to a file.I send the rtp stream using ffmpeg as follows : ffmpeg -i "D:\highway.avi" -s 176x144 -vcodec mpeg4 -f rtp rtp://127.0.0.1:5000/
I got an error So I specified the audio as Nil using the following command : -an ffmpeg -i "D:\highway.avi" -s 176x144 -an -vcodec mpeg4 -f rtp rtp://127.0.0.1:5000/Now it is sending data to RTP successfully.
Using RTP receiver filter, I received the RTP data and dumped it to the file using Dump Filter.
I passed the RTP dump as input to the ffmpeg as follows :
ffmpeg -vodec mpeg4 -f rtp -i "D:\RTP.dump" -vcodec rawvideo "D:\RTP.YUV"
I played it with YUV file player, with some flickering it plays the RTP Mpeg data well.

ffmpeg RTP streaming

ffmpeg -vcodec mpeg4 -i D:\Media\vel1.mp4 -an -s 176x144 -vcodec rawvideo -f rtp rtp://127.0.0.1:5000/

while running this in a commandline, we got the following things in ffmpeg command informations.


I copied the following things in a rawvideo.sdp file and opened it in a
QuickTime player. Now the video is playing.


SDP:( this content will be generated by ffmpeg)
--------
v=0
o=- 0 0 IN IPV4 127.0.0.1
t=0 0
s=No Name
a=tool:libavformat
c=IN IP4 127.0.0.1
m=video 5000 RTP/AVP 96
a=rtpmap:96 MP4V-ES/90000
a=fmtp:96 profile-level-id=1