{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Bifacial Capacity Test with CapTest Class\n",
    "\n",
    "This example shows use of the CapTest class to conduct a capacity test for a plant with bifacial modules by substituting POA irradiance for total POA irradiance. The CapTest class provides an entry point to define the type of test (regression formula) and define other test level parameters like the test tolerance, reporting conditions, and bifaciality.\n",
    "\n",
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import inspect\n",
    "import warnings\n",
    "# warnings.filterwarnings('ignore')\n",
    "from IPython.display import display, Markdown\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "# import captest as pvc\n",
    "import captest as ct\n",
    "from captest import capdata as pvc\n",
    "from captest.captest import CapTest\n",
    "from bokeh.io import output_notebook, show\n",
    "\n",
    "# uncomment below two lines to use cptest.scatter_hv in notebook\n",
    "import holoviews as hv\n",
    "hv.extension('bokeh')\n",
    "\n",
    "#if working offline with the CapData.plot() method may fail\n",
    "#run 'export BOKEH_RESOURCES=inline' at the command line before\n",
    "#running the jupyter notebook\n",
    "\n",
    "output_notebook()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Data and Calculate Regressors\n",
    "\n",
    "By specifying the `test_setup` as `bifi_e2848_etotal` when instantiating the CapTest instance the $E_{total}$ regressors are calculated, added as columns to `data` and `data_filtered`, and the `regression_columns` is updated to map `poa` to `e_total`. These steps are documented by default in output printed."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "## uncomment line below to review options for pre-defined capacity tests\n",
    "# ct.captest.TEST_SETUPS.keys()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "## uncomment line below to review the full docstring with all parameters for the CapTest class\n",
    "# display(Markdown(f\"```\\n{inspect.getdoc(CapTest)}\\n```\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts = CapTest.from_params(\n",
    "    test_setup='bifi_e2848_etotal',\n",
    "    meas_path='./data/example_meas_data_bifi.csv',\n",
    "    sim_path='./data/pvsyst_example_HourlyRes_2_bifi.CSV',\n",
    "    test_tolerance='+/- 3',\n",
    "    ac_nameplate=6_000,\n",
    "    bifaciality=0.7,\n",
    "    meas_load_kwargs={\n",
    "        'group_columns': './data/column_groups_bifi.xlsx',\n",
    "    },\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.meas.agg_sensors(agg_map={'real_pwr_inv':'sum'}, verbose=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note, the full functionality of the dashboard requires a live notebook. Try installing to run or using the launch binder button at the top of the page. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "combine = {'inv_sum_mtr_pwr': ['mtr', 'inv.*agg'], 'irr_all':['irr_poa', 'irr_ghi'], 'temp_all':['temp_amb', 'temp_mod']}\n",
    "default_groups = ['inv_sum_mtr_pwr', 'irr_all', 'temp_all']\n",
    "ts.meas.plot(combine=combine, default_groups=default_groups)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Filtering Measured Data\n",
    "The `CapData` class provides a number of convience methods to apply filtering steps as defined in ASTM E2848.  The following section demonstrates the use of the more commonly used filtering steps to remove measured data points.\n",
    "\n",
    "The `regression_formula` and `regression_cols` attributes are printed below for review prior to fitting the regression to enable verifying the regression is fitted against the $E_{total}$ irradiance."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.meas.regression_formula"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.meas.regression_cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Uncomment and run to copy over the filtered dataset with the unfiltered data.\n",
    "ts.meas.reset_filter()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "measured_filters = [\n",
    "    (pvc.CapData.filter_custom, (pd.DataFrame.dropna, ), {}),\n",
    "    (pvc.CapData.filter_irr, (ts.min_irr, ts.max_irr), {}),\n",
    "    (pvc.CapData.filter_outliers, (), {}),\n",
    "    (pvc.CapData.fit_regression, (), {'filter':True, 'summary':False}),\n",
    "    (pvc.CapData.rep_cond, (), {}),\n",
    "    (pvc.CapData.filter_irr, (ts.rep_irr_filter_low, ts.rep_irr_filter_high), {'ref_val':'rep_irr'}),\n",
    "    (pvc.CapData.fit_regression, (), {}),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pvc.run_test(ts.meas, measured_filters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.meas.get_summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(\n",
    "    ts.meas.scatter_filters().opts(tools=['lasso_select']) +\n",
    "    ts.meas.timeseries_filters().opts(width=1000)\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ct.plotting.ScatterPlot(cd=ts.meas, timeseries=True, tc_power=True, tc_mode='replace', split_day=True).view()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load and Filter PVsyst Data\n",
    "\n",
    "To load and filter the modeled data, often from PVsyst, we simply create a new CapData object, load the PVsyst data, and apply the filtering methods as appropriate."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To load pvsyst data we use the `load_data` method with the `load_pvsyst` option set to True.  By default the `load_data` method will search for a csv file that includes `pvsyst` in the filename in a `data` directory in the same directory as this file.  If you have saved the pvsyst file in a different location, you can use the `path` and `fname` arguments to load it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.sim.regression_formula"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.sim.regression_cols"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Write over cptest.flt_sim dataframe with a copy of the original unfiltered dataframe\n",
    "ts.sim.reset_filter()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.min_irr"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "simulated_filters = [\n",
    "    (pvc.CapData.filter_time, (), {'test_date':'10/11/1990', 'days':ts.sim_days}),\n",
    "    (pvc.CapData.filter_irr, (ts.min_irr, 930), {}),\n",
    "    (pvc.CapData.filter_pvsyst, (), {}),\n",
    "    (pvc.CapData.filter_irr, (ts.rep_irr_filter_low, ts.rep_irr_filter_high), {'ref_val':ts.meas.rc['poa'][0]}),\n",
    "    (pvc.CapData.fit_regression, (), {}),\n",
    "]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "pvc.run_test(ts.sim, simulated_filters)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.sim.get_summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.sim.scatter_filters()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Results\n",
    "\n",
    "The `get_summary` and `captest_results_check_pvalues` functions display the results of filtering on simulated and measured data and the final capacity test results comparing measured capacity to expected capacity, respectively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.get_summary()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.meas.rc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.meas.print_points_summary(ts.hrs_req)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.captest_results_check_pvalues()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The `overlay_scatters` function can be used to overlay the final scatter plots from scatter plots of all filtering steps produced above."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ts.overlay_scatters()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}