The two tests, i.e, correlator tests for stable fringes and receiver
deflection tests to find out whether the receivers show expected
deflections are important and fundamental. Even when all antennas are
working satisfactorily, correlator and the receiver tests are essential,
particularly to compare this with forthcoming feed rotation to same
frequency band. It is preferable for these tests to be done after the
pointing correction.
Preparing
for the tests
The tests are to be run on when all antennas have been released
after maintenance. Near the end of the weekly maintenance, usually a few
antennas will not be in the working state either due to extended
maintenance or due to some problem. A list of non-working antennas, as
per the controlroom should be made around 3 pm. This list should be
compared with the GCC to find out which are the antennas are under
maintenance and which one has genuine problem. This will help to get
back these antennas along with the antennas given for maintenance. It
is preferable to wait till all antennas are released after the
maintenance before starting the tests.
Pointing
It is recommended that after the weekly maintenance, both elevation
and azimuth pointings be done. It should be done after loading zero
offsets in respective axis. Load zero elevation offsets, run the
elevation pointing, update the elevation pointing. Load zero azimuth
pointing and similarly do the azimuth pointing. These values make a good
database for FPS calibration.
Correlator
Tests
First look at the matmon on a strong point
source (3C48/3C147/3C286). Record a short stretch of lta
and ltb files with shorter integration (say 8, 4 or
2s for ten min or so). Run tax on this and check
the phase and amplitude. Some of the possible problems are
One or more antennas do not show fringes. Investigate why the
antenna is 'dead'. Check (i) whether first LO is locked (ii) 30to1
output is sensible (iii) status at other sideband, other channel (iv)
correlator bandshape etc
(i) When the first LO shows unlock: Change the LO settings to
255 and come back to the required LO. Try setting slightly different
LOs. If nothing works, raise a callsheet and followup.
(ii) 30to1 output is too low: Check the attenuations, and follow
this with receiver tests.
(iii) The other sideband is working!: If the 30to1 is OK and
only one sideband does not show fringes then the problem is localised to
either baseband or correlator. Check the baseband output, if it is OK,
then the problem is localised to correlator. Run corr_config and try.
If nothing works raise a callsheet and followup.
(iv) If the correlator bandshape is not satisfactory, run
corr_config. Check if this antennas has different clock_sel.
One or a few antennas shows amplitude jumps similar to full
scintillation. Record the data and check the bandshapes. If 130 and 175
bandshapes are showing identical selfs suspect delay rack. Check
the same for other sideband also. Run newdly_config, if the problem
persists, call for expert help..
If the If 130 and 175 bandshapes are similar and highly
correlated, but not identical, it is likely to be due to LO, discuss
with engineers.
A set of antennas (usually in blocks of four, five or ten,
depends on where the problem is) not showing fringes.. Run
newdly_config and corr_config. If the problem does not go, call for
expert help. Also check whether the tube lights are ON in the
correlator room, ensure that they are OFF.
Some baselines showing very high cross or 9999.
Run corr_config and check. Also check whether there is saturation by
checking correlator and 30to1 bandshapes.
Antenna showing '0' for cross and self. Such problems usually
comes if (i) the antenna is not set, (ii) baseband power too low or
(iii) the corresponding cable is not connected to sampler in the
correlator room. Inform GCC. Also check the baseband output, and the
connectivity from baseband to correlator.
(i) The antenna is not set: Run the settings once more.
(ii) baseband power too low: Check the baseband and 30to1
output. If the baseband power is too low in one channel, check the
connection from baseband to correlator.
(iii) Cable problem: Check the cable inside correlator room.
130 and 175 self bandshapes identical. This is to do with
delay rack, run newdly_config.
Antenna bandshape replicated, channels 64 to 127 are replica of 0
to 63. Run newdly_config.
Some of the problems with delay rack does not go with
newdly_config and require power reset. This should be done only
by the correlator group or GCC.
Recording a short stretch (10 min) data with 128 milli-second
integration on a few channels (say, every 16th channel) will also be
useful. Check self power as a function of time, fringes, dropouts, etc.
Ionospheric scintillations sometimes confuse with the correlator
problems. Recording a small stretch of data on a strong calibrator and
looking at the amplitude and phase will help to clarify. It is possible
to differentiate between ionospheric and correlator problems.
Scintillations usually dominant at arm antennas (long baselines) and
shortest baselines should continue to give reasonable fringes. In an
given arm, the amplitude of phase change will increase with the baseline
length in case of ionospheric problems. For example, C09-W03 should
show larger but similar phase fluctuations than C09-W01 and C09-W06
should exhibit even larger, similar fluctuations. In the case of
correlator problems, it will be independent of the baseline length.
Receiver
Tests
Receiver tests are essential to find out whether the antennas give
the desired amount of deflection against a strong source. This needs
30to1 to check the deflection on strong sources (CYGA, CASA, CRAB) and
can be done when the correlator is not available. The second sensitivity
test is done using pulsar mode of the correlator and uses relatively
weak sources (point sources with flux of a few tens of Jy will do).
Even when all antennas are working satisfactorily, the deflection tests
are essential, particularly to compare this with forthcoming feed
rotation to same frequency band.
Checking the authif table:
This is a simple and short experiment. Point the antenna to the cold
sky., i.e, to a calibrator source not stronger than 30 Jy or so.. Run
corresponding authif. Wait for two minutes. Record 1 dump at 30to1 and
run 'difftable' on this file. This will give a list of antennas which
significantly deviate from authif values. If the difference is more than
6 dB, the reason why such a large difference needs investigation. Raise
a callsheet.
30to1 deflection tests:
The better way of doing this experiment is by doing power
equalisation at the 30to1 output on strong source. Eventhough using
authif values gives reasonable power equalisation, on strong sources a
deviation of 4 dB from the operating point could lead to saturation and
could lead to wrong interpretation of low sensitivity. Therefore to
reduce the scatter, it is recommended to power equalise at the 30to1
output to a value of 50 dBm on a strong source.
Set the IF pre-and post attenuation to large value (say 20 + 20 dB).
While running power equalisation programme, please note that there is a
delay of about 1.5 to 2 minutes in updating the power levels at the
30to1 display, after the attenuations have been changed. Hence, if the
power equalisation is done too quickly, it will never converge.
After the power equalisation to 50dBm, record 3
dumps on source and 3 dumps off source. For off source, go to a cold
part of the sky without disturbing the attn settings. It is safer to go
to a weak VLA calibrator (flux about a Jy or less).
If less than half an hour is available, this experiment can also be
done by running authif +8 or 12 dB as per requirement (if the
expected deflection is around 8 dB add 8, if 12 dB, add 12). In this
case there is no need to do power equalise at 30to1 output. Go to strong
source. Record 3 dumps per antenna for all antennas on source and off
source. Sometimes this does not equalise to 50dBm
at 30to1 for all antennas (on source), hence for those antennas where
the power levels differ significantly from 50dBm,
the results will be less reliable. While interpreting 'poor' sensitivity
results, please look at the on source power level to find out about the
possibility of saturation.
Run the programme 'getdefl'. To run this, login
to tcc-l1 as 'elab' and move to directory '/data/cspa/data'.
Make a directory for the day like 'ddmmmyy' (eg: 01jan05). Move the
30to1 data files to this directory if it is located elsewhere.
'getdefl' will give two output files, one PS file and one ASCII table
giving the deflection values. The PS file contains all the
informations, like bandshape, deflection, on-source power, off-source
power etc. Check whether the deflection you get is as expected.. Look
for some of the following problems.
Some antennas do not show proper deflection. Check pointing, if
required re-focus the feed. If no improvement, FES/ABR is to be
suspected. If only one channel is dead, try swapping. If the dead
channel swaps, the problem is before swap-switch, if does not swap, the
problem is after swap-switch, in the IF chain. You can also try to see
the deflection with noise cal.
Some antennas show ripples in the passband, particularly at
higher frequencies. This is known as Elevation Dependent Oscillations
(EDOs). Note down the azimuth and elevation..
Bad bandshapes (slope, dips etc). Try RF swap to check whether
the problem is before before swap-switch or not. This could also be due
to gain mismatch at OTx and/or ORx.
Spurious spikes. Repeat the experiment with FE terminated (with
attenuations adjusted appropriately to bring the power level as close to 50dBm
as possible). If the spike is present even after terminating FE, it is
generated internally, mostly at the antenna base. Further test can be
done by changing LO by 5 MHz step and checking whether the spike moves
by times 5 MHz. These can be done by GCC, while investigating
further.
Bring these to the notice of GCC. Most of the above problems can
also be seen in the spectrum analyser kept in the Receiver Room. For
problematic antennas, go to Receiver Room, check the bandshapes in the
spectrum analyser independently. This will help to rule out any possible
error introduced by 30to1 switch.
Correlator deflection tests:
This procedure allows the user to get a quick and useful estimate of
the basic parameters (like Ta/Tsys, Mean/RMS and S/N) of each antenna
from a 10 minute observation of a standard calibrator source,
using correlator data output. The steps are as follows:
Track one of the standard calibrator sources of moderate strength
e.g. 3c286, 3c48, 3c147 (even some extended sources like 3c353 can be
used); however very strong sources like VIRGO, CYGNUS, Crab etc should
NOT be used for this experiment.
Configure correlator in standard pulsar mode: corr_config -conf
pulsar, ALC off (NOTE: if fstop has been running, just killing it is NOT
sufficient: a fresh corr_config -conf pulsar HAS TO BE DONE to erase the
effects of the last fringe and fstc settings!)
Do power equalisation of antenna signals (preferably from inside
pulsar_console).
In pulsar console, select "Ant Cal Run" option from within the
"Antenna" pop-up sub-menu. Set the correct cal source name and "Save" -
this should update the appropriate file (calibration.set in
/temp2/data/astro/pulsar on bhaskar) which has the sequence of commands
to move the antennas OFF src (+/- 5 deg in decl) and ON src from the
calibrator.
Set the name of the file in which you want to record the
correlator self data e.g. /pulsar2/psr_sw/Tcltk/SCRATCH/cal_test.dat.
NOTE : this is NOT a lta/b file! Set the "Duration" of the recording to
be 8 minutes.
In Online, open the SAC file
/temp2/data/astro/pulsar/calibration.set and start execution of it.
When the antennas have just moved from ON source to OFF source
for the first time in the run, start the recording of the correlator
data, using "Start Run" in the "Calibration Setup" window. The
acquisition into the file will automatically stop after the specified
duration is over. Source goes to off source.
The recorded data can be monitored (even while the acquisition is
going on) and also analyzed using the "Plot Cal Run" option in the
"Antenna" window. Here, data from any number of antennas can be plotted
simultaneously, for any selected frequency channel of the correlator.
Further, the transitions between the ON and OFF regions ("edges") can be
identified and the signal levels for the ON and OFF regions can be
computed from which quantities like Mean/RMS, Ta/Tsys and S/N for each
antenna can be computed and plotted as a function of channel number.
These allow for easy monitoring of the basic health parameter of each
antenna.
ALL TCLTK program are available in astro2:/pulsar2/psr_sw/Tcltk/GUI.
Login as pulsar and type tcl (this should take you to the required
directory).
Callsheets
Whenever the problems are confirmed, callsheet should be generated with
full information of RF settings, what all have been tried from control
room before giving up.
Troubleshooting
While the tests are going on smoothly, some of the following
problems can occur.
MSEB power reset: Some antennas, after MSEB power reset,
gives intermittent time outs. Give reset through mcm 0. If all mcm
time out, give ABC reset through telephone. If nothing works, inform
GCC. Sometimes, antennas also go off due to high voltages. Inform GCC
and wait for the voltage to come down.
RF Switch Operation: The usual medicine is to put 30+30 dB
attenuations or FE termination. Sometimes these does not work. Try
changing LO to a very different value (say, to the default 1st LO of
another RF band). Telephone reset also can be tried. One of them will
get the antenna back.
Command failed: Issue the command again. Note down which
COMH the command failed. If occurs frequently, call GCC. It is known
that command no. '0' fails, which comes with a cycle of 145 commands.
Radio Frequency Interference (RFI): Check which are all the
antenna show strong RFI lines. Is there a pattern? Are these antennas
close to housing? Controlled experiments can be done by the RFI team in
coordination with MSEB whether RFI goes down of the MSEB power is
switched off at certain locations.
Also refer to the attached document by K. Sanjay "No Fringe, or
Fringe Vanished? Try This!".
Please send your comments to
ishwar@ncra.tifr.res.in
No Fringe, or Fringe
vanished? Try This!
This document is based on the experience from control room. More
points can be added by individuals if they have, which are not covered
in this document.
No Self?
Make sure that RF, LO, IF & BB has been set properly, if
not set it again.
Make sure the bandshapes, from correlator o/p to see the
baseband is properly set and the channel used (in matmon) to see fringe
has proper power in it. Some time baseband fails to set one set
of antenna and all such antennas remains at old settings. But this
happens for 4 antennas (one pol[130 or 175] only) at a given time.
If self at corr o/p is zero, and BB power is OK, try noise in
correlator and make sure correlator is giving right o/p power. If it
is, there is likely to be a problem with connecting cables from BB to
CORR. If correlator is not being used for observation, self test can be
done.
Self ok but no
fringe?
Set LO and confirm LO LOCK.(LO monitoring need to make robust!)
Check if existing pointing offsets are not loaded.
Find out whether feed is not rotated or is in acceptable range
of counts given by theodolite experiment.
If it is just start of new Frequency after long time. make sure
swap/unswap has been taken into account, try swapping RF in such antenna
and update the RF-swap table accordingly.
If calibrator is being observed, a small pointing can be done
to see if the there is pointing offset by loading different
offsets w.r.t. the existing one, rather than taking antenna out of
subarry. Same test can be done in case antenna is giving low fringe
amplitude. User need to load pointing offsets in steps of the beamwdth
of current Freq. in observation, which is usefull to save time. This
happens mostly if feed is rotated for some reason after the last
pointing was done.
Make sure antenna/ABC/servo takes the proper source co-ordinate
(target) and track accordingly. 'sacw' window in ondisp(ONLINE)
should always show zero error below that antenna for elevation
and azimuth while antenna is in tracking.
If simultaneously five antennas are failed, and they are from
same FFT control card, try corr_config, or let correlator
expert attend it.
If only CSQ antennas are showing fringes and not arm antennas,
it means either 'fstop' is not running or das has been given
wrong source or somehow 'fringe stop' has failed.
If self power keeps on fluctuating, or showing constant high
power, it is likely to be a RFI in that channel only and in that antenna
only (in case the RFI is generated in system of same antenna).
check the bandshape, and matmon in other channel.
If fringe amplitude
is low
Check phases on calibrator. If phases are winding, find out if
any other antenna which shows similar winding. If such a pair
exists, check the connectivity of cables in OF/BB/corr or
mapping in corrsys.hdr, for whether they have got swapped. If
there is such a pair for which there is no fringe amplitude and phase
for no clear reason, and the pair form longer baseline, the problem is
most likely to be of this sort.
If fringe amplitude is low and phase is showing slightly more
or no scatter, then check if the RF noise has left on.
Same, if fringe amplitude is low and phase also showing
somewhat more scatter, check if the antenna need RF swap and that
antenna has more polarization leakage, so that the fringe
amplitude user get is not real but leaked.
If fringe vanished
Suddenly:
Check if servo has applied the brakes, OR ABC has got reset.
Check if that antenna is showing 'flat bandshape' at correlator
output.
If fringe falls gradually, it's most likely that, ONLY ABC got
reset and other electronic, viz. RF, LO, IF, are still ok, but
servo stopped tracking as effect of ABC reset.
Check if RF switch has got operated, in that case, ABC starts
giving ABC T.O. and self power in matmon falls to very small value and
30-1 monitoring also show the fall in power leve.
Check if feed has got rotated, few antennas had problem of feed
rotation on power on reset.(this test can be ignored if such problem
does not exists.)
Make sure the antenna setting was not got changed (by mistake)
with some other antenna under maintainance.
If fringe in all antennas has gone down by some extent, it's
more likely that, while loading pointing offset in one antenna,
the same offset has got loaded in all antenna by mistake.
If fringe in all antennas suddenly stopped, check the master LO
is OK, or correlator is getting STA as well as GPS clock (using
irq_seq)
Check if LO1 has been become unlock. (more technical
clarification needed here, for can this really happen)
If all antennas
showing no (or unstable) fringe:
check if proper CLK_SEL was selected in 'init.hdr.new' file and
the same is given to 'dlytrk' with a option '-clk_sel'. Also
make sure appropriate BB bandwidth selected.
Make sure, by using 'showband' that the channel selected for
seeing fringe, in matmon is not affected by any unstable
features like, RFI or band oscillations.
Watch if the error in 'acq30' o/p is showing 'processing lagged
behind'. As effect of this record does not print the 'number of
STA cycles' (under title "wt") same as given (e.g. while in no
error condition, if LTA1=16 & LTA2=8, the wt should be equal to 128,
a product of LTA1 & LTA2, in error condition this number will be
lesser than that.)
For individual antennas, if fringe started showing unstability,
check if that antenna has started showing EDO.
If all antennas showing fluctuation in phases as well as fringe
amplitude, it's well known ionosperic scintillation.