{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Concise Example Capacity Test using pvcaptest\n",
    "\n",
    "This example performs the same test as the Complete Capacity Testing example, but uses the `run_test` function to apply the filters and then displays the filtering steps visually using the `scatter_filters` method.\n",
    "\n",
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import warnings\n",
    "warnings.filterwarnings('ignore')\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "# import captest as pvc\n",
    "import captest as ct\n",
    "from captest import capdata as pvc\n",
    "from bokeh.io import output_notebook, show\n",
    "\n",
    "# uncomment below two lines to use cptest.scatter_hv in notebook\n",
    "import holoviews as hv\n",
    "from holoviews import opts\n",
    "hv.extension('bokeh')\n",
    "\n",
    "#if working offline with the CapData.plot() method may fail\n",
    "#run 'export BOKEH_RESOURCES=inline' at the command line before\n",
    "#running the jupyter notebook\n",
    "\n",
    "output_notebook()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and Plot Measured Data\n",
    "Load the measured data with the `load_data` method, which returns a CapData object. This example uses a column grouping defined in an excel file. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "das = ct.load_data('./data/example_meas_data.csv', group_columns='./data/column_groups.xlsx')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "das.column_groups"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "das.regression_cols = {\n",
    "    'power': 'real_pwr_mtr',\n",
    "    'poa': ('irr_poa', 'mean'),\n",
    "    't_amb': ('temp_amb', 'mean'),\n",
    "    'w_vel': ('wind_speed', 'mean')\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "das.process_regression_columns()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "das.agg_sensors(agg_map={'real_pwr_inv':'sum'})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note, the full functionality of the dashboard requires a live notebook. Try installing to run or using the launch binder button at the top of the page. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "combine = {'inv_sum_mtr_pwr': ['mtr', 'inv.*agg'], 'irr_all':['irr_poa', 'irr_ghi'], 'temp_all':['temp_amb', 'temp_mod']}\n",
    "default_groups = ['inv_sum_mtr_pwr', 'irr_all', 'temp_all']\n",
    "das.plot(combine=combine, default_groups=default_groups)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Filtering Measured Data\n",
    "The `CapData` class provides a number of convience methods to apply filtering steps as defined in ASTM E2848.  The following section demonstrates the use of the more commonly used filtering steps to remove measured data points."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment and run to copy over the filtered dataset with the unfiltered data.\n",
    "das.reset_filter()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "measured_filters = [\n",
    "    (pvc.CapData.filter_custom, (pd.DataFrame.dropna, ), {}),\n",
    "    (pvc.CapData.filter_irr, (200, 2000), {}),\n",
    "    (pvc.CapData.filter_outliers, (), {}),\n",
    "    (pvc.CapData.fit_regression, (), {'filter':True, 'summary':False}),\n",
    "    (pvc.CapData.rep_cond, (), {}),\n",
    "    (pvc.CapData.filter_irr, (0.5, 1.5), {'ref_val':'self_val'}),\n",
    "    (pvc.CapData.fit_regression, (), {}),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pvc.run_test(das, measured_filters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "das.get_summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "das_scatter = das.scatter_filters()\n",
    "das_scatter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "das.timeseries_filters().opts(width=1200)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and Filter PVsyst Data\n",
    "\n",
    "To load and filter the modeled data, often from PVsyst, we simply create a new CapData object, load the PVsyst data, and apply the filtering methods as appropriate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To load pvsyst data we use the `load_data` method with the `load_pvsyst` option set to True.  By default the `load_data` method will search for a csv file that includes `pvsyst` in the filename in a `data` directory in the same directory as this file.  If you have saved the pvsyst file in a different location, you can use the `path` and `fname` arguments to load it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sim = ct.load_pvsyst('./data/pvsyst_example_HourlyRes_2.CSV')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sim.column_groups"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sim.set_regression_cols(power='real_pwr__', poa='irr_poa_', t_amb='temp_amb_', w_vel='wind__')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write over cptest.flt_sim dataframe with a copy of the original unfiltered dataframe\n",
    "sim.reset_filter()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "simulated_filters = [\n",
    "    (pvc.CapData.filter_time, (), {'test_date':'10/11/1990', 'days':60}),\n",
    "    (pvc.CapData.filter_irr, (200, 930), {}),\n",
    "    (pvc.CapData.filter_pvsyst, (), {}),\n",
    "    (pvc.CapData.filter_irr, (0.5, 1.5), {'ref_val':das.rc['poa'][0]}),\n",
    "    (pvc.CapData.fit_regression, (), {}),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pvc.run_test(sim, simulated_filters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sim.get_summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "sim_scatter = sim.scatter_filters()\n",
    "sim_scatter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results\n",
    "\n",
    "The `get_summary` and `captest_results_check_pvalues` functions display the results of filtering on simulated and measured data and the final capacity test results comparing measured capacity to expected capacity, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts = ct.CapTest(meas=das, sim=sim, ac_nameplate=6_000, test_tolerance='+/- 7')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.get_summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "das.rc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.captest_results_check_pvalues(print_res=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Overlaying scatter plots from the measured and PVsyst data. This plot can be generated using `CapTest.overlay_scatters` when running a test using an instance of `CapTest`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    das.scatter_hv().relabel('Measured') *\n",
    "    sim.scatter_hv().relabel('PVsyst')\n",
    ").opts(\n",
    "    opts.Scatter(alpha=0.3, width=600)\n",
    ")"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}