Add a PSTH similarity metric for quality assurance

[abuzarmahmood]

Can be added as a second part to units_similarity
Could be as simple as a pearson correlation of concatenated psths for all tastes
Use ephys_data.py to get firing rates, then average for each taste to get PSTH

[abuzarmahmood]

Summary of Suggestions and Changes

1. Files Identified for Modification:

   • ephys_data.py: This file contains the ephys_data class, which handles electrophysiology data and includes methods for calculating firing rates. It is essential for generating PSTHs (Peri-Stimulus Time Histograms).
   • unit_similarity.py: This file focuses on analyzing unit similarity in neural spike data and can be extended to include a PSTH similarity metric.

2. Plan to Address the Issue:

   • Extract Firing Rates: Use the get_firing_rates method in ephys_data.py to obtain firing rates.
   • Compute PSTHs: Average the firing rates for each taste to generate PSTHs.
   • Calculate Similarity: Implement a function in unit_similarity.py to compute the Pearson correlation of concatenated PSTHs for all tastes.

3. Specific Changes Suggested:

   • Compute PSTHs in ephys_data.py:

     • File: /home/exouser/Desktop/blech_github_bot/repos/katzlabbrandeis/blech_clust/utils/ephys_data/ephys_data.py
     • Line: After line 862
     • Description: Add a method to compute PSTHs by averaging firing rates for each taste.
     • Code Snippet:

       def compute_psths(self):
           """
           Compute PSTHs by averaging firing rates for each taste.
           """
           if 'firing_list' not in dir(self):
               self.get_firing_rates()
           
           self.psths = [np.mean(firing, axis=0) for firing in self.firing_list]

   • Add PSTH Similarity Metric in unit_similarity.py:

     • File: /home/exouser/Desktop/blech_github_bot/repos/katzlabbrandeis/blech_clust/utils/qa_utils/unit_similarity.py
     • Line: After line 102
     • Description: Implement a function to calculate the Pearson correlation of concatenated PSTHs.
     • Code Snippet:

       def psth_similarity(psths):
           """
           Calculate the Pearson correlation of concatenated PSTHs for all tastes.
           
           Args:
               psths: List of PSTHs for each taste.
           
           Returns:
               similarity_matrix: A matrix of Pearson correlation coefficients.
           """
           concatenated_psths = np.concatenate(psths, axis=1)
           similarity_matrix = np.corrcoef(concatenated_psths)
           return similarity_matrix

   • Integrate PSTH Similarity Calculation:

     • File: /home/exouser/Desktop/blech_github_bot/repos/katzlabbrandeis/blech_clust/utils/qa_utils/unit_similarity.py
     • Line: Inside the main execution block (after line 245)
     • Description: Integrate the PSTH similarity calculation into the main execution flow.
     • Code Snippet:

       # Assuming `data` is an instance of `ephys_data`
       data.compute_psths()
       psth_similarity_matrix = psth_similarity(data.psths)
       print("PSTH Similarity Matrix Calculated")

These changes will implement the requested PSTH similarity metric for quality assurance, leveraging existing functionalities in the repository.

---

This response was automatically generated by blech_bot using model gpt-4o

[abuzarmahmood]

Also add:
1- a threshold for psth similarity in sorting_params
2- a plot to plot both the raw psth similarity matrix and the thresholded one in subplots
3- warnings to unit similarity output file

[abuzarmahmood]

Updated Suggestions and Changes

1. Files Identified for Modification:

   • ephys_data.py: This file contains the ephys_data class, which handles electrophysiology data and includes methods for calculating firing rates. It is essential for generating PSTHs (Peri-Stimulus Time Histograms).
   • unit_similarity.py: This file focuses on analyzing unit similarity in neural spike data and can be extended to include a PSTH similarity metric.

2. Plan to Address the Issue:

   • Extract Firing Rates: Use the get_firing_rates method in ephys_data.py to obtain firing rates.
   • Compute PSTHs: Average the firing rates for each taste to generate PSTHs.
   • Calculate Similarity: Implement a function in unit_similarity.py to compute the Pearson correlation of concatenated PSTHs for all tastes.
   • Add Threshold and Plotting: Introduce a threshold for PSTH similarity and plot both the raw and thresholded similarity matrices.
   • Warnings: Add warnings to the unit similarity output file when thresholds are exceeded.

3. Specific Changes Suggested:

   • Compute PSTHs in ephys_data.py:

     • File: /home/exouser/Desktop/blech_github_bot/repos/katzlabbrandeis/blech_clust/utils/ephys_data/ephys_data.py
     • Line: After line 862
     • Description: Add a method to compute PSTHs by averaging firing rates for each taste.
     • Code Snippet:

       def compute_psths(self):
           """
           Compute PSTHs by averaging firing rates for each taste.
           """
           if 'firing_list' not in dir(self):
               self.get_firing_rates()
           
           self.psths = [np.mean(firing, axis=0) for firing in self.firing_list]

   • Add PSTH Similarity Metric in unit_similarity.py:

     • File: /home/exouser/Desktop/blech_github_bot/repos/katzlabbrandeis/blech_clust/utils/qa_utils/unit_similarity.py
     • Line: After line 102
     • Description: Implement a function to calculate the Pearson correlation of concatenated PSTHs.
     • Code Snippet:

       def psth_similarity(psths, threshold):
           """
           Calculate the Pearson correlation of concatenated PSTHs for all tastes.
           
           Args:
               psths: List of PSTHs for each taste.
               threshold: Threshold for similarity.
           
           Returns:
               similarity_matrix: A matrix of Pearson correlation coefficients.
               thresholded_matrix: A matrix with values above the threshold.
           """
           concatenated_psths = np.concatenate(psths, axis=1)
           similarity_matrix = np.corrcoef(concatenated_psths)
           thresholded_matrix = np.where(similarity_matrix > threshold, similarity_matrix, np.nan)
           return similarity_matrix, thresholded_matrix

   • Integrate PSTH Similarity Calculation and Plotting:

     • File: /home/exouser/Desktop/blech_github_bot/repos/katzlabbrandeis/blech_clust/utils/qa_utils/unit_similarity.py
     • Line: Inside the main execution block (after line 245)
     • Description: Integrate the PSTH similarity calculation into the main execution flow and plot the results.
     • Code Snippet:

       # Assuming `data` is an instance of `ephys_data`
       data.compute_psths()
       psth_similarity_matrix, thresholded_matrix = psth_similarity(data.psths, similarity_cutoff)
       
       # Plotting
       fig, (ax1, ax2) = plt.subplots(1, 2, figsize=(12, 5))
       im1 = ax1.matshow(psth_similarity_matrix, cmap='viridis')
       plt.colorbar(im1, ax=ax1)
       ax1.set_title('Raw PSTH Similarity Matrix')
       
       im2 = ax2.matshow(thresholded_matrix, cmap='hot')
       plt.colorbar(im2, ax=ax2)
       ax2.set_title(f'Thresholded PSTH Similarity Matrix (>{similarity_cutoff})')
       
       plt.tight_layout()
       plt.savefig(os.path.join(output_dir, 'psth_similarity_matrix.png'))
       plt.close()
       
       # Warnings
       if np.any(thresholded_matrix > similarity_cutoff):
           with open(warnings_file_path, 'a') as f:
               print('PSTH similarity threshold exceeded.', file=f)

These changes will implement the requested PSTH similarity metric for quality assurance, leveraging existing functionalities in the repository, and address the additional feedback regarding thresholds, plotting, and warnings.

---

This response was automatically generated by blech_bot using model gpt-4o

[abuzarmahmood]

[ develop_issue ]

[abuzarmahmood]

Created pull request: #559
Continue discussion there.

---

This response was automatically generated by blech_bot using model gpt-4o