As agreed during the webmeeting on 9 April, this forum thread is for discussion of relevant issues and proposals related to the measurement and post processing methodology for the Iridium NEXT satellite constellation.

Please see the documents from the 9 April webmeeting for the relevant background information.

Regards,

Peter

Dear all,

please find here below a proposal from Thorsten Senger (Bnetza) on the sofware package, in particular for the Data reduction.   For any question/comment and further discussion, please use this Forum.  I also note Thorsten email address:  [email protected]

In the subscript "flux_distribution" exists a part, where the standard deviation is calculated from the background-measurement and the measurement itself is masked with this value:

> % Estimate rms for the measurement from that of the background values

> % rms_S=std(Sb)*sqrt(size(Sb,1)/size(S,1));

> %

> % mask all values < rms_S

> %

> for i1=1:size(S,1) S0(i1,:)=S(i1,:).*(S(i1,:)>rms_S) ; end;

 The factor sqrt(size(Sb,1)/size(S,1)) should be eliminated from the script, because the background measurement got the same integration time as the measurement from the satellite and does not need to be adjusted regarding the different recording times.

 So the new command lines would be:

 > %

> rms_S=std(Sb);

> %

> % mask all values < rms_S

> %

  for i1=1:size(S,1) S0(i1,:)=S(i1,:).*(S(i1,:)>rms_S) ; end;

> %

 This change does not affect the "Measurement Reports" from the "DataReduction" package at all.

It is a lower limit for the values which will be used for the EPFD-Calculation.

It does also not significantly change the Results from EPFD-calculation in the past, because these values where clearly higher than this value range, but it should be corrected before further calculations.

Dear all,

in the meantime I thought about alternative solutions to the baseline problem
with the new IRIDIUM Next Leeheim measurements. Thorsten Senger kindly provided
me measured raw data of three examples scans. I have now a much better idea of
the difficulties we are faced with when processing future observations.

I think, a better solution to tackle the problem would be a data-centric
approach, where we use measurements itself to estimate the background level
caused from changes in the atmosphere and ground radiation (with elevation).
The biggest challenge here is the separation of IRIDIUM signals from the noise
floor. If this could be done with sufficient accuracy, one could feed the
true IRIDIUM intensity into subsequent calculations. Note, that personally, I'd
still be in favor of applying a distance correction before producing the CDFs.

I played a little bit with the data and, even without putting too much effort
into it, I came up with an algorithm that seems to work with sufficient
accuracy. The IRIDIUM OOB features are relatively smooth in frequency but have
high temporal variability, i.e., if one plots the data vs. time, there are lots
of "spikes". A running-median filter can suppress these spikes quite
efficiently. I achieved even better results by applying the median filter
iteratively, flagging outliers in each iteration (which are excluded in the
next filtering step). The outlier detection in the residuals ("data -
filtered") can also be improved by smoothing in frequency with different kernel
widths.

My solution is not lightning fast - it takes about 10 minutes to process one
scan (single-threaded) - but this should be sufficient, because one only needs to
run this once for each scan.

I attach several plots, which show some of the results. The spectrograms show
the raw data (linearized), the estimated baseline using the above algorithm,
the outlier flag mask (a by-product of the algorithm), and the residual. I
omitted the colorbars as the data is not calibrated for intensity. For the
residual plot, the blue-white-red colorbar is symmetric (i.e., zero = white).
Note that for the flag mask, some features can be missing in the plot due to
aliasing. (I didn't want to send you an image file with a width of more than
12000 pixels to cover the full time range.) If one zooms in, only the weakest
RFI signals are missing in the flag mask.

One could also use the flag mask to restrict the follow-up analysis to affected
pixels. This way, one would also better handle the sensitiviy problem, which we
briefly discussed in the last meeting. As an example, I created histograms of
all pixel intensities within the flag mask and not within the mask. The latter
yields a relatively clean Normal distribution, while the former has a tail, as
expected. Not surprisingly, around zero intensity there is still a certain
fraction of the noise floor distribution. It may be viable to fit a Normal-
distribution to the negative side and subtract this from the CDF accordingly.

If you're interested in the details, I can provide a Jupyter (Python) notebook
and a PDF printout of it on request. For now, I attach some figures with the
result. I'm going to participate in the meeting on Friday, where I could also
give further explanations.

Best regards,
Benjamin

Dear all,

attached you will find a PowerPoint document concerning tests we made to

eliminate elevation dependent noise from iridium satellite measurements with

elevation dependent background measurements we made.

Best regards,
Leeheim

(Thorsten)

Thosten / Benjamin - can you please make the underlying data from your scans available please?  It would be helpful for everyone to examine the same data.

Many thanks in advance,

Mike

Dear Mike,

since, I don't own the data, I'd prefer the Leeheim people (e.g., Thorsten) share the data with you. However, I attach the Python script that I created such that you could have a closer look into the algorithm, which I developed.

Best,

Benjamin

Dear all,

please find attached the output from our last dedicated meeting in Maisons-Alfort (27 April 2018). It is a working document to trace the progress on the discussions and also lists the actions to be taken towards resolving the different issues currently identified.

regards,

Soraya C

Dear Thorsten, all,

Thanks for your responses - I will try to address some below, but others may require a further document or discussion at our July meeting. As you will note, some questions require the response of parties who are not yet part of this email chain. I've asked the SE40 chair to establish a reflector or forum so we capture the exchange for all to follow.

Also, can you kindly reference the document when you offer comments? - Iridium has provided two documents for discussion on this subject (SE40(23)012, and SE40(23)0xx to be renumbered).

1. Evaluation with DataReduction Software:

• Iridium agrees that the comparison of results will need further discussion in order to identify the differences in values. During our processing, we used a few different “assumptions” but were unable to replicate the Leeheim data exactly (these assumptions are not documented in the BNetzA measurement report). The results in Table 1 (SE40(23)012) were made (I believe) with the same combination of measurement-files, background-files, and calibration files as you kindly provided in the report and in earlier exchanges – but I will confirm this.
• Iridium agrees that we should examine a single measurement in more detail, to identify the difference in outcome.
• (N/A – Iridium believes it used the same approach)
• Note that Iridium was not able to replicate the max (mean_SJy) values for each pass at the every first stage (see 3.1, SE40(23)0xx), or the calibrated files from the raw files (section 3.3, SE40(23)0xx), indicating a problem in this first stage of processing.
• Finally, please note that we expect the overall result will be slightly different, as we combined the different measurements differently for reasons highlighted in SE40(23)0xx, including balancing the choice of orbit planes.

1. Evaluation with EPFD Software:

• The original scripts were developed by Prof Jessner (MPIfR), and later developed by Thales/ANFR within SE40.
• I need to consult others to answer your detailed questions here, and I hope to provide an answer ahead of our meeting.

1. When is the goal reached?:

• I’m afraid the Iridium document has given the wrong impression, when I read your statement: The [Iridium] document says that the EPFD evaluation does not fulfil the requirements, because there is a disproportion between day and night measurements and even/odd planes. Iridium believes the measurements are valid and useful, but we need to agree how to use them. We expect a difference between day and night, and between even and odd planes – this is a reflection of the traffic on respective satellites, and hence the out-of-band signals generated. However, we need to agree within SE40 how to generate a comparable average that will be useful moving forward, and also how future measurements are taken.
• Even given the points we identified, the overall results in the Leeheim calculations and the Iridium calculations are close (6.6dB vs 6.1dB).
• Within our discussion, Iridium hopes to (i) identify the source of the differences (particularly in point (1) above), (ii) find consensus to record the results in an ECC Report, and (iii) agree on the specification of future measurements. We are not advocating further measurements now, but we anticipate further satellite upgrades which will require measurement, perhaps later in 2023/24.

I hope this helps clarify the issues. Our meeting in July will help to move this along, but we can also exchange further information as needed.

Best regards,

Mike

-----Original Message-----
From: [email protected] <[email protected]>
Sent: Thursday, June 15, 2023 3:10 PM
To: Michael Thompson <[email protected]>; [email protected]; [email protected]; [email protected]
Cc: Jaime Afonso <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; Jaime Afonso <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]
Subject: AW: EXTERNAL:information document - Report on IRIDIUM unwanted emissions 1610.6 – 1613.8 MHz - 2022

Dear all, dear Mike,

I would like to coment/question some oft he points in your document!

--------------------------------------------

1. Evaluation with DataReduction Software:

Regarding the differences in the results of the DataReduction package (thresholds exceeded), the document shows a table with differences in the Iridium<->Leeheim calculated results (Table 1: Summary of the satellites observed in the 2022 measurement campaign)

In general it isabsolutely necessary, that when presenting results, you explain in detail how you arrived at those results. In the same way we do it in our Leeheim reports. Otherwise nobody can validate your results !!!

In relation to your Table 1: "Summary of the satellites observed in the 2022 measurement campaign", I must therefore specifically ask:

!!!            Did you use exactly the same combination of measurement-files, background-files, and calibration files as shown in our report with exactly the same parameters (meas-times, backg-times, star fuxes  etc)                 ???

If you can confirm this, I think we should pick a single measurement and calculate it in parallel to find out why and where these differences occur.

If you not used exactly the same files/parameters as shown in our report, you should exactly present how you did your calculation !!!

--------------------------------------------

2 Evaluation with EPFD Software:

Concerning the EPFD evaluation i also would like generally to say something:

Some time ago these script where modified. I don't know who did this.

As a result the calculation was splitted in two parts: OOB-evaluation and Noise-evaluation.

I could not find anything in the existing recommendation about why the same algorithm should not be used for both evaluations (OOB and Noise).

Or otherwise:      !!!! What is the difference between these two calculations and as a conclusion ----> which additional measurements have to be performed to feed the noise part of the EPFD-scripts ???

Does the Noise-part (1583)

- require only a point-backgound measurement ?,

- does it require a hemisphere measurement without satellite ?

- does it require a elevation corrected hemisphere measurement without satellite ?

   (which means you need 2 noise-measurements with additionell point background measurement)

I assumed the latter to be the case, but it is not described anywhere, and there is no corresponding requirement or measurement specification for the Leeheim measurements concerning this.

Also it must be clear how many of those noise measurements the EPFD-Noise package needs, and when those measurements have to be made.

The document says :   "  ... we noted that a few noise passes did not have a background spot measurement" !!! 

Of course they do not, because for evaluating the measurements with the DataReduction Packet, you do not need this. And you not needed it fort he "old" EPFD evaluation.

If it is now needed, we should have been informed before we did the measurements - > after reprogramming the EPFD software, a new measurement specification has obviously not been created to this day.

This was the reason why we did use old noisepass measurements with addional backgound point measurements which we performed coincidentally in the past.

In my opinion, piecing together a file (using a 25 second window from the measurement) like you did is not a good solution.

Furthermore if you be aware what the EPFD Noise-Part exatly does and what input it needs, you need to carefully examine the uncertaintys of the EPFD SW by using Leeheim measurements.

I have already pointed out that even with small changes in the average of the input values (e.g. 20 Jy), which is definitely within the measurement inaccuracy of our measuring station, the resulting DEC factor varies by 3 dB when considering the EPFD noise.

The measurement uncertainty always increases as the measured values move towards the sensitivity of the system.

In case oft he Noise-measurements we try to measure the sensitivity itself.

The current EPFD-Noise package seem to significantly reinforces this uncertainty.

All of these points must be examined and clarified before you can believe to get a trusting result from the EPFD evaluation.

I don't see this now, and a comparison to old EPFD-evaluation is definitly not reliable.

--------------------------------------------

3. When is the goal reached?

The SE40 should also clarify what is the ultimate point concerning the question, when the distortions are acceptable.

The document says that the EPFD evaluation does not fulfill the requirements, because there is a disproportion between day and night measurements and even/odd planes...

I agree with that point. But to get rid of this problem, there has to be made an measuement-campaign with a proper amount of measurements from all planes in the same amount for day and night. Which goes along withs an enormous measurement campaign with lots of night-time measurements and dependend on the constraints (measure same satellites day night , min elev, at which day and night hours...) it would also take a very long time to get satellites of all the planes and get all this done.

If the exceeding tresholds produced by the Data-reduction package is a point that has to be fulfilled, then in the first point it is only nesseccary to look at the satellites with the higest distortions, That means day-measurements of even planes. Odd planes and night measurements are not of importance in this view.

Or should it be possible that the EPFD-evaluation (if it is working proper someday) fulfill the requirements, while single measurements from the DataReduction evaluation does not !?

--------------------------------------------

As far as all this is not cleared i only see reliable and comparable results in the evaluation of the exeeding tresholds using the DataReduction part.

So i definitely agree with the summary in this document:  "...further development of a specification for OOB measurements Is proposed."

A further development of a specification for OOB measurements especially for the EPFD calculation is absolutely necessary!

Best regards

Thorsten

Thorsten Senger

_____________________________________

Bundesnetzagentur für Elektrizität, Gas, Telekommunikation, Post und Eisenbahnen Satellitenmessstelle 64560 Riedstadt-Leeheim

Tel.         (06158)940 - 209

PC Fax: (0 18 05) 73 48 70 - 16 78

mailto:[email protected]

http://www.BNetzA.de

Dear Mike, dear all,

As my email was a direct answer to your email from 14.06.2023 containing the document SE40(23)0xx

it should have been unmistakably clear that my comments are referred to this document you send in this email.

Thanks for your answer, but it did not help to clarify issues. The opposite is more the case.

You wrote:   "... During our processing, we used a few different “assumptions”..."

What assumptions you used, how did you process your results ???

I'm sorry, Mike, but it can't work like that.

Iridium can not always just present results without detailed reports about how they evaluated their results.

We are in no position to have to ask you for this information every time again.

We always provide detailed reports about how we did our evaluation on basis of our measurement.

And you know if you claime missing information, we are always happy to provide it to you at any time.

If you want to discuss the differences in our results (even if the differences are relativ small) you have to deliver the same

evaluation reports the way we do. This is not a one-way procedure !!!

For every of your results i expect a report from you containing all information about what files you used to feed the scrpits, which times

you have choosen ... and of course the reports the scripts automaticly produce. The same way we allways do.

Otherwise it makes no sense to discuss the different results.

------

Concerning the EPFD evaluation i don't want to repeat the issues i wrote in my email from 15.06.2023 again.

But as long as this questions are not answered, there is no basis for a further discussion.

------

Without these reports and information from you by the end of this month, I see no point in a meeting and it should be postponed.

Best regards

Thorsten

-----Ursprüngliche Nachricht-----

Von: Michael Thompson <[email protected]>

Gesendet: Freitag, 16. Juni 2023 16:49

An: Esch16-4 <[email protected]>; 231-1 <[email protected]>; 231-3 <[email protected]>; [email protected]

Cc: Jaime Afonso <[email protected]>; 231 <[email protected]>; Esch16-2 <[email protected]>; Esch16 <[email protected]>; Esch16-5 <[email protected]>; Jaime Afonso <[email protected]>; [email protected]; [email protected]; 223a <[email protected]>; 223-3 <[email protected]>; Gary Rees <[email protected]>

Betreff: RE: EXTERNAL:information document - Report on IRIDIUM unwanted emissions 1610.6 – 1613.8 MHz - 2022

Dear Thorsten, all,

Thanks for your responses - I will try to address some below, but others may require a further document or discussion at our July meeting. As you will note, some questions require the response of parties who are not yet part of this email chain. I've asked the SE40 chair to establish a reflector or forum so we capture the exchange for all to follow.

Also, can you kindly reference the document when you offer comments? - Iridium has provided two documents for discussion on this subject (SE40(23)012, and SE40(23)0xx to be renumbered).

1. Evaluation with DataReduction Software:

*             Iridium agrees that the comparison of results will need further discussion in order to identify the differences in values. During our processing, we used a few different “assumptions” but were unable to replicate the Leeheim data exactly (these assumptions are not documented in the BNetzA measurement report). The results in Table 1 (SE40(23)012) were made (I believe) with the same combination of measurement-files, background-files, and calibration files as you kindly provided in the report and in earlier exchanges – but I will confirm this.

*             Iridium agrees that we should examine a single measurement in more detail, to identify the difference in outcome.

*             (N/A – Iridium believes it used the same approach)

*             Note that Iridium was not able to replicate the max (mean_SJy) values for each pass at the every first stage (see 3.1, SE40(23)0xx), or the calibrated files from the raw files (section 3.3, SE40(23)0xx), indicating a problem in this first stage of processing.

*             Finally, please note that we expect the overall result will be slightly different, as we combined the different measurements differently for reasons highlighted in SE40(23)0xx, including balancing the choice of orbit planes.

2. Evaluation with EPFD Software:

*             The original scripts were developed by Prof Jessner (MPIfR), and later developed by Thales/ANFR within SE40.

*             I need to consult others to answer your detailed questions here, and I hope to provide an answer ahead of our meeting.

3. When is the goal reached?:

*             I’m afraid the Iridium document has given the wrong impression, when I read your statement: The [Iridium] document says that the EPFD evaluation does not fulfil the requirements, because there is a disproportion between day and night measurements and even/odd planes. Iridium believes the measurements are valid and useful, but we need to agree how to use them. We expect a difference between day and night, and between even and odd planes – this is a reflection of the traffic on respective satellites, and hence the out-of-band signals generated. However, we need to agree within SE40 how to generate a comparable average that will be useful moving forward, and also how future measurements are taken.

*             Even given the points we identified, the overall results in the Leeheim calculations and the Iridium calculations are close (6.6dB vs 6.1dB).

*             Within our discussion, Iridium hopes to (i) identify the source of the differences (particularly in point (1) above), (ii) find consensus to record the results in an ECC Report, and (iii) agree on the specification of future measurements. We are not advocating further measurements now, but we anticipate further satellite upgrades which will require measurement, perhaps later in 2023/24.

I hope this helps clarify the issues. Our meeting in July will help to move this along, but we can also exchange further information as needed.

Best regards,

Mike

Dear Thorsten et al,

I acknowledge the frustration you have expressed in your message. My Iridium colleagues have also felt frustration trying to resolve the differences between our results, despite your kind assistance in providing the many data files BNetzA produced. However, I hope you recognise that the processing is not deterministic - there are options not only in the processing (start/stop times, CasA ref, etc) but also in the combination of satellites to simulate. As you probably recall, we have shared results and discussed these points within SE40 during every previous measurement campaign (since 2005!), before finding consensus and proceeding to publish the results in a report. I hope our meeting will address this, amongst other issues (see below).

You requested that Iridium provides one set of our processed files, to allow a closer examination to identify divergence. My colleagues are preparing this now, and I hope to share with you shortly. (Although we have not identified the source, there is some suspicion that we may be using different software versions, and so I will also provide the software version we have used so you can verify it against your version.)

Under work item SE12, SE40 has responsibility for measurement and ongoing monitoring of the Iridium/RAS compatibility issue. Iridium has made improvements to the satellite configuration since 2019, to reduce the out-of-band emissions, and results have been promising. We have also been engaged in separate discussions with radio astronomers at the Max Plancke Institute (MPIfR), to improve future compatibility. The results of these discussions will benefit the radio astronomy community as a whole, but we anticipate further measurement campaigns (under work item SE12) will be necessary to quantify and confirm improvements. For this reason, Iridium hopes that our meeting will also address this aspect, for which your input will be gratefully received.

I also expressed to Jesus that I anticipate that one meeting may not be sufficient. Recognising that we have a full SE40 meeting in September, we may have email exchange (or another virtual meeting) to prepare for that meeting. But that is subject to mutual agreement, of course. A verbal exchange is essential to start this work, in my view, especially noting the frustrations felt alround.

Best regards,

Mike

On 20/06/23 10:32 Thorsten Senger wrote:
"

Dear Mike, dear all,

As my email was a direct answer to your email from 14.06.2023 containing the document SE40(23)0xx

it should have been unmistakably clear that my comments are referred to this document you send in this email.

Thanks for your answer, but it did not help to clarify issues. The opposite is more the case.

You wrote:   "... During our processing, we used a few different “assumptions”..."

What assumptions you used, how did you process your results ???

I'm sorry, Mike, but it can't work like that.

Iridium can not always just present results without detailed reports about how they evaluated their results.

We are in no position to have to ask you for this information every time again.

We always provide detailed reports about how we did our evaluation on basis of our measurement.

And you know if you claime missing information, we are always happy to provide it to you at any time.

If you want to discuss the differences in our results (even if the differences are relativ small) you have to deliver the same

evaluation reports the way we do. This is not a one-way procedure !!!

For every of your results i expect a report from you containing all information about what files you used to feed the scrpits, which times

you have choosen ... and of course the reports the scripts automaticly produce. The same way we allways do.

Otherwise it makes no sense to discuss the different results.

------

Concerning the EPFD evaluation i don't want to repeat the issues i wrote in my email from 15.06.2023 again.

But as long as this questions are not answered, there is no basis for a further discussion.

------

Without these reports and information from you by the end of this month, I see no point in a meeting and it should be postponed.

Best regards

Thorsten

-----Ursprüngliche Nachricht-----

Von: Michael Thompson <[email protected]>

Gesendet: Freitag, 16. Juni 2023 16:49

An: Esch16-4 <[email protected]>; 231-1 <[email protected]>; 231-3 <[email protected]>; [email protected]

Cc: Jaime Afonso <[email protected]>; 231 <[email protected]>; Esch16-2 <[email protected]>; Esch16 <[email protected]>; Esch16-5 <[email protected]>; Jaime Afonso <[email protected]>; [email protected]; [email protected]; 223a <[email protected]>; 223-3 <[email protected]>; Gary Rees <[email protected]>

Betreff: RE: EXTERNAL:information document - Report on IRIDIUM unwanted emissions 1610.6 – 1613.8 MHz - 2022

Dear Thorsten, all,

Thanks for your responses - I will try to address some below, but others may require a further document or discussion at our July meeting. As you will note, some questions require the response of parties who are not yet part of this email chain. I've asked the SE40 chair to establish a reflector or forum so we capture the exchange for all to follow.

Also, can you kindly reference the document when you offer comments? - Iridium has provided two documents for discussion on this subject (SE40(23)012, and SE40(23)0xx to be renumbered).

1. Evaluation with DataReduction Software:

*             Iridium agrees that the comparison of results will need further discussion in order to identify the differences in values. During our processing, we used a few different “assumptions” but were unable to replicate the Leeheim data exactly (these assumptions are not documented in the BNetzA measurement report). The results in Table 1 (SE40(23)012) were made (I believe) with the same combination of measurement-files, background-files, and calibration files as you kindly provided in the report and in earlier exchanges – but I will confirm this.

*             Iridium agrees that we should examine a single measurement in more detail, to identify the difference in outcome.

*             (N/A – Iridium believes it used the same approach)

*             Note that Iridium was not able to replicate the max (mean_SJy) values for each pass at the every first stage (see 3.1, SE40(23)0xx), or the calibrated files from the raw files (section 3.3, SE40(23)0xx), indicating a problem in this first stage of processing.

*             Finally, please note that we expect the overall result will be slightly different, as we combined the different measurements differently for reasons highlighted in SE40(23)0xx, including balancing the choice of orbit planes.

2. Evaluation with EPFD Software:

*             The original scripts were developed by Prof Jessner (MPIfR), and later developed by Thales/ANFR within SE40.

*             I need to consult others to answer your detailed questions here, and I hope to provide an answer ahead of our meeting.

3. When is the goal reached?:

*             I’m afraid the Iridium document has given the wrong impression, when I read your statement: The [Iridium] document says that the EPFD evaluation does not fulfil the requirements, because there is a disproportion between day and night measurements and even/odd planes. Iridium believes the measurements are valid and useful, but we need to agree how to use them. We expect a difference between day and night, and between even and odd planes – this is a reflection of the traffic on respective satellites, and hence the out-of-band signals generated. However, we need to agree within SE40 how to generate a comparable average that will be useful moving forward, and also how future measurements are taken.

*             Even given the points we identified, the overall results in the Leeheim calculations and the Iridium calculations are close (6.6dB vs 6.1dB).

*             Within our discussion, Iridium hopes to (i) identify the source of the differences (particularly in point (1) above), (ii) find consensus to record the results in an ECC Report, and (iii) agree on the specification of future measurements. We are not advocating further measurements now, but we anticipate further satellite upgrades which will require measurement, perhaps later in 2023/24.

I hope this helps clarify the issues. Our meeting in July will help to move this along, but we can also exchange further information as needed.

Best regards,

Mike

".