DataProcessor-v2.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Iron-Python Notebook for Processing Raw Hardware Results"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This will be an all-in-one package for processing data/results of workload execution archives in specific directories in this repository.\n",
    "It is assumed that all the results were captured using WorkloadExec-v2.py file, so that everything is archived appropriately.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## About fields and its interpretation\n",
    "\n",
    "Before explaining the fields, its essential to understand how data is gathered & sources.\n",
    "\n",
    "<img src=\"report/phase-2/rsrc/Experiment-setup.drawio.png\" width=\"50%\">\n",
    "\n",
    "As shown in the above image, there are two data sources in this experiment setup\n",
    "1. **ODroid-XU4 hardware:** which is running the workload sent by development PC \n",
    "\t- This provides two sets of data:\n",
    "\t\t1. **Perf-stat** data captured while the worload executed, with a sampling interval of 100ms. Details about the features(perf counters) being sampled is discussed later.\n",
    "\t\t1. **Polled Data**: To capture data from ODroid board which are not available in perf-list. This includes thermal, CPU usage statistics etc.\n",
    "1. **SmartPower3 Power Monitor:** which provides the total power drawn by ODroid XU-4 hardware\n",
    "\n",
    "### Time Stamp information\n",
    "\n",
    "From the above mentioned source/sampling method, each record will be having UTC timestamp - using which this data will be collated.\n",
    "\n",
    "|     Field Name    |               Description                                                                     |\n",
    "|-------------------|-----------------------------------------------------------------------------------------------|\n",
    "|  utctime          |  UTC Time field - output of mergig of above mentioned datasets.                               |\n",
    "|  utctime_x        |  UTC Time from perf-stat data added for reference only and is an output of table merging.     |\n",
    "|  utctime_y        |  UTC Time from power monitor data added for reference only and is an output of table merging. |\n",
    "|  ts_utc           |  UTC Time from polled data added for reference only and is an output of table merging.        |\n",
    "\n",
    "\n",
    "\n",
    "### Perf-stat data \n",
    "\n",
    "The columns corresponding to Perf-stat command output of a workload will be like the ones listed below.\n",
    "\n",
    "```console\n",
    "...\n",
    "S0-D0-C0_branch-instructions\n",
    "S0-D0-C0_branch-misses\n",
    "S0-D0-C0_branch-load-misses\n",
    "S0-D0-C0_branch-loads\n",
    "S0-D0-C0_bus-cycles\n",
    "S0-D0-C0_cpu-cycles\n",
    "S0-D0-C0_instructions\n",
    "S0-D0-C0_cache-misses\n",
    "S0-D0-C0_cache-references\n",
    "S0-D0-C0_cpu-clock\n",
    "S0-D0-C0_L1-dcache-load-misses\n",
    "S0-D0-C0_L1-dcache-loads\n",
    "S0-D0-C0_L1-dcache-store-misses\n",
    "S0-D0-C0_L1-dcache-stores\n",
    "S0-D0-C0_L1-icache-load-misses\n",
    "S0-D0-C0_L1-icache-loads\n",
    "S0-D0-C0_LLC-load-misses\n",
    "S0-D0-C0_LLC-loads\n",
    "S0-D0-C0_LLC-store-misses\n",
    "S0-D0-C0_LLC-stores\n",
    "...\n",
    "...\n",
    "```\n",
    "\n",
    "This is  following a naming template like so:\n",
    "```\n",
    "<Silicon-#>-<Die-#>-<Core-#>_<perf-field-name>\n",
    "```\n",
    "So a fieldname prefix of S0-D0-C0 implies core-1 of the CPU on the chip. Similarly, you will find \n",
    "- S0-D0-C0 ~ S0-D0-C3\n",
    "- S1-D0-C0 ~ S1-D0-C3\n",
    "wherein S0 and S1 will correspond to Little and Big cluster with  C0~C3 cores per cluster.\n",
    "\n",
    "More info: https://man7.org/linux/man-pages/man1/perf-stat.1.html#:~:text=%2D%2Dper%2Dcore%20Aggregate%20counts,(system%2Dwide). (search for -per-core)\n",
    "\n",
    "\n",
    "Now on to the field with names after:\n",
    "\n",
    "|        Field Name          |               Description                             |\n",
    "|----------------------------|-------------------------------------------------------|\n",
    "|  branch-instructions       |  Number of branch instructions executed               |\n",
    "|  branch-misses             |  Number of branch missed                              |\n",
    "|  branch-load-misses        |  Number of load misses                                |\n",
    "|  branch-loads              |  Number of branch loads                               |\n",
    "|  bus-cycles                |  Bus cycle counted                                    |\n",
    "|  cpu-cycles                |  CPU Cycle counted                                    |\n",
    "|  instructions              |  Number of instructions executed                      |\n",
    "|  cache-misses              |  Number of cache misses                               |\n",
    "|  cache-references          |  Number of Cache references                           |\n",
    "|  cpu-clock                 |  CPU clocks utilized                                  |\n",
    "|  L1-dcache-load-misses     |  Number of L1 Data Cache load misses                  |\n",
    "|  L1-dcache-loads           |  Number of L1 Data cache loads                        |\n",
    "|  L1-dcache-store-misses    |  Number of L1 Data Cache store misses                 |\n",
    "|  L1-dcache-stores          |  Number of L1 Data cache stores                       |\n",
    "|  L1-icache-load-misses     |  Number of L1 instruction cache load misses           |\n",
    "|  L1-icache-loads           |  Number of L1 instruction cache loads                 |\n",
    "|  LLC-load-misses           |  Number of Last-level-cache (L2) cache load misses    |\n",
    "|  LLC-loads                 |  Number of Last-level-cache (L2) cache loads          |\n",
    "|  LLC-store-misses          |  Number of Last-level-cache (L2) cache store misses   |\n",
    "|  LLC-stores                |  Number of Last-level-cache (L2) cache stores misses  |\n",
    "\n",
    "\n",
    "### Polled data \n",
    "\n",
    "These fields are captured by polling at a regular interval while workload is executing, by sampling sysfs file (/sys/devices/virtual/thermal/thermal_zone[0~3]/temp)\n",
    "\n",
    "|        Field Name    |               Description                             |\n",
    "|----------------------|-------------------------------------------------------|\n",
    "|  therm_cpu0          |  Thermal reading reported by CPU thermal region#0     |\n",
    "|  therm_cpu1          |  Thermal reading reported by CPU thermal region#1     |\n",
    "|  therm_cpu2          |  Thermal reading reported by CPU thermal region#2     |\n",
    "|  therm_cpu4          |  Thermal reading reported by CPU thermal region#3     |\n",
    "\n",
    "**Note/Correction**: therm_cpu4 is mislabled as 4, instead it is 3 corresponding to thermal_zone3 values\n",
    "\n",
    "\n",
    "### Power Monitor data \n",
    "\n",
    "|        Field Name          |               Description                                                                        |\n",
    "|----------------------------|--------------------------------------------------------------------------------------------------|\n",
    "|  dev_ippwr-ch1-volts_mV    |  Voltage(mV) supplied on to channel-1 of SM3 power supply - on to which ODroid-XU4 is connected. |\n",
    "|  dev_ippwr-ch1-ampere_mA   |  Current(mA) drawn by the connected board from channel-1.                                        |\n",
    "|  dev_ippwr-ch1-watt_mW     |  Power(mW) drawn by the connected board from channel-1.                                          |\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Imports"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "\n",
    "# General Packages\n",
    "import os\n",
    "import sys\n",
    "import glob\n",
    "import io\n",
    "import shutil\n",
    "## For handling BZ archive of result files\n",
    "import requests\n",
    "import tempfile\n",
    "import tarfile\n",
    "\n",
    "# For data processing \n",
    "import datetime\n",
    "import re\n",
    "import csv\n",
    "import pandas\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Utility Functions"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "def extract_data(data_archive:str, tmp_working_dir: str) ->str:\n",
    "    # TempDirectory to string returns something like ' <TemporaryDirectory ' prefixed\n",
    "    ## for handling tmp directory cleanly, striping off these characters\n",
    "    tmpfilePath = str(tmp_working_dir).split(' ')[1].replace('\\'','').replace('>','')\n",
    "    # print('Temporary directory used: '+tmpfilePath+'/')\n",
    "\n",
    "    # Extract Files\n",
    "    archive = tarfile.open(data_archive, 'r:bz2')\n",
    "    # An assumption is made here that the archives are stored at directory level, than directly files\n",
    "    # Hence getting the first level directory name \n",
    "    # e.g.: <TarInfo '11-03-2023_14-58-03_BigCore-10itr-50msSmplg-CPUFreq-2.0GHz' at 0x7fbc676ecd00>\n",
    "    dirname=str(archive.getmembers()[0]).split(' ')[1].replace('\\'','')\n",
    "\n",
    "    archive.extractall(tmpfilePath+'/')\n",
    "    archive.close()\n",
    "    return os.path.join(tmpfilePath,dirname)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Function to process Perf-stat output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Define regular expressions to match the lines with data\n",
    "# Regular expression for [1] & [2], and associated result variable(s)\n",
    "re_1_2_firstlevel = re.compile(r\"\"\"\n",
    "                                ^    # Line start\n",
    "                                [#]+ # Immediate occurance of '#'\n",
    "                                .*   # Anything following that\n",
    "                            \"\"\", re.X)\n",
    "re_1_timestamp_info = re.compile(r\"\"\"\n",
    "                                ^[#]+                 # Line Start with # character\n",
    "                                \\s+started\\s+on\\s+    # This string will be present before timestamp\n",
    "                                (.*)                  # Time of perf start\n",
    "                                \"\"\", re.X)\n",
    "\n",
    "\n",
    "# Regular expression for [4]\n",
    "re_4_summary_header = re.compile(r\"\"\"^\\s+\n",
    "                                Performance\\scounter\\sstats\\sfor\\s\n",
    "                                \\'(.*)\\'\\s+              # Type of monitoring in perf\n",
    "                                \\((\\d+)\\sruns\\):\n",
    "                                \"\"\",re.X)\n",
    "\n",
    "# Regular expression for [3]\n",
    "re_3_perf_stat_record = re.compile(r\"\"\"^\\s+\n",
    "                                (\\d+\\.\\d+)\\s+          # Timestamp\n",
    "                                (\\S+)\\s+(\\d+)\\s+       # CPU-ID, CPUs\n",
    "                                (.*)                   # counts, [unit], events, event-info\n",
    "                                \"\"\", re.X)\n",
    "re_3_subrec_countfield_nc = re.compile(r\"\"\"\n",
    "                                    <not\\scounted>\\s+     # Is it a <not counted field> ?\n",
    "                                    (.*)\n",
    "                                    \"\"\", re.X)\n",
    "re_3_subrec_countfield = re.compile(r\"\"\"\n",
    "                                    (\\d+[.\\d]*)\\s+        # counts\n",
    "                                    (.*)              # units or event-name\n",
    "                                    [#]+(.*)                  # Event extra info\n",
    "                                    \"\"\", re.X)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "def Process_ProfFile(proffile:str, outcsvfile:str) -> None:\n",
    "    with io.open(proffile, 'rt') as  proffile_entry:\n",
    "        \n",
    "        stats_gatherer = {}\n",
    "        perf_stat_starttime = ''\n",
    "        perf_stat_starttime_dateobj = None\n",
    "        perf_stat_currtime_dateobj = None\n",
    "        perf_stat_columns = []\n",
    "        perf_stat_runtype=''\n",
    "        perf_stat_runcount=0\n",
    "\n",
    "\n",
    "        records_ended = False\n",
    "        record_counter = 0\n",
    "        prev_timestamp =''\n",
    "        rec_timestamp = rec_cpuid = rec_cpucnt = rec_counter = rec_eventname = ''\n",
    "        csvout = open(outcsvfile, 'w') \n",
    "        csvwriter = None\n",
    "        #Decoding the prof/perf-stat file data\n",
    "        for line in proffile_entry.readlines():\n",
    "            # print (str(len(line))+':'+line)\n",
    "\n",
    "            ## Okay so, its a line starting with # character\n",
    "            if re_1_2_firstlevel.match(line):\n",
    "                # is it a time stamp info line??\n",
    "                re_1_match = re_1_timestamp_info.match(line)\n",
    "                if (re_1_match):\n",
    "                    perf_stat_starttime = re_1_match.group(1)\n",
    "                    perf_stat_starttime_dateobj = datetime.datetime.strptime(perf_stat_starttime,'%a %b %d %H:%M:%S %Y')\n",
    "                    perf_stat_starttime_pddateobj = pandas.Timestamp(perf_stat_starttime_dateobj, unit='ns')\n",
    "                    perf_stat_currtime_dateobj = datetime.datetime.strptime(perf_stat_starttime,'%a %b %d %H:%M:%S %Y')\n",
    "                    perf_stat_currtime_pddateobj = pandas.Timestamp(perf_stat_currtime_dateobj)\n",
    "                    # print('Start Time:'+str(perf_stat_currtime_pddateobj))\n",
    "                else:\n",
    "                    # Then it must be a column header info line\n",
    "                    perf_stat_columns = line[1:].split()\n",
    "            else:\n",
    "                # So its lines other than meta info ones ([1] & [2])\n",
    "                # just to skip the last few records as we only need the perf-stat samples, not the summary at the end\n",
    "                if (records_ended == False):\n",
    "                    re_4_match = re_4_summary_header.match (line)\n",
    "                    re_3_match = re_3_perf_stat_record.match(line)\n",
    "                \n",
    "                    if (re_4_match):\n",
    "                        perf_stat_runtype  = re_4_match.group(1)\n",
    "                        perf_stat_runcount = int(re_4_match.group(2))\n",
    "                        records_ended = True\n",
    "                        # Okay, so we have reached till the summary section\n",
    "                        # write the last record\n",
    "                        record_counter += 1\n",
    "                        csvwriter.writerow(stats_gatherer)\n",
    "                        stats_gatherer.clear()\n",
    "                        \n",
    "                    elif (re_3_match):\n",
    "                        rec_timestamp = re_3_match.group(1)\n",
    "                        rec_cpuid     = re_3_match.group(2)\n",
    "                        rec_cpucnt    = re_3_match.group(3)\n",
    "\n",
    "\n",
    "                        if (prev_timestamp==''):\n",
    "                            prev_timestamp = rec_timestamp\n",
    "                            ns = perf_stat_starttime_pddateobj+pandas.Timedelta(seconds=float(rec_timestamp))\n",
    "                            # print('PDTime:'+str(ns)+' subsec: '+rec_timestamp)\n",
    "                            stats_gatherer['utctime'] = ns\n",
    "\n",
    "                        if (prev_timestamp != rec_timestamp):\n",
    "                            ## Okay, we are encountering a new record, so dump the old one\n",
    "                            record_counter += 1\n",
    "                            if (record_counter == 1):\n",
    "                                csvwriter =  csv.DictWriter(csvout, stats_gatherer.keys())\n",
    "                                csvwriter.writeheader()\n",
    "                            csvwriter.writerow(stats_gatherer)\n",
    "                            stats_gatherer.clear()\n",
    "                            prev_timestamp = rec_timestamp\n",
    "\n",
    "                            ns = perf_stat_starttime_pddateobj+pandas.Timedelta(seconds=float(rec_timestamp))\n",
    "                            # print('PDTime:'+str(ns)+' subsec: '+rec_timestamp)\n",
    "                            stats_gatherer['utctime'] = ns\n",
    "\n",
    "                        re3__ncsubrec_match = re_3_subrec_countfield_nc.match(re_3_match.group(4))\n",
    "\n",
    "                        if (re3__ncsubrec_match):\n",
    "                            trec = re3__ncsubrec_match.group(1)\n",
    "                            # print (rec_cpuid + ' NC - '+trec)\n",
    "                            stats_gatherer[rec_cpuid+'_'+trec] = 'NaN'\n",
    "                            pass\n",
    "                        else:\n",
    "\n",
    "                            re_3_subrec_match = re_3_subrec_countfield.match(re_3_match.group(4))\n",
    "                            if (re_3_subrec_match):\n",
    "                                split_str =re_3_subrec_match.group(2).split() \n",
    "                                # print('event: '+re_3_subrec_match.group(1)+'-'+ split_str[len(split_str)-1] )\n",
    "                                rec_counter = re_3_subrec_match.group(1)\n",
    "                                rec_eventname = split_str[len(split_str)-1]\n",
    "                            else:\n",
    "                                ## Special cases, some instructions stats count seems to be having no '#' separator\n",
    "                                ## Handle  it separately. e.g:\n",
    "                                ##     808323      instructions                                                         (3.64%)\n",
    "                                ##     52451588      instructions                                                         (8.98%)\n",
    "                                ##     424251      instructions                                                         (8.92%)\n",
    "                                ##     394060      instructions                                                         (8.86%)\n",
    "                                tstr = re_3_match.group(4)\n",
    "                                rec_counter   = tstr[0]\n",
    "                                rec_eventname = tstr[1]\n",
    "                                assert len(tstr) != 3,'Some invalid records structure found, please check!!!'\n",
    "                                # print(re_3_match.group(4))\n",
    "\n",
    "                            if (rec_eventname == 'cpu-clock'):\n",
    "                                stats_gatherer[rec_cpuid+'_'+rec_eventname] = float(rec_counter)\n",
    "                            else:\n",
    "                                stats_gatherer[rec_cpuid+'_'+rec_eventname] = int(rec_counter)\n",
    "                                \n",
    "                else:\n",
    "                    ## For now, we are only going to process the individual records, not the summary\n",
    "                    ## So we can very well break out of the loop\n",
    "                    break\n",
    "\n",
    "\n",
    "        # print ('Start time: '+ perf_stat_starttime)\n",
    "        # print ('Columns: '+ str(perf_stat_columns))\n",
    "        # print ('runs type: '+ perf_stat_runtype)\n",
    "        # print ('runs count: '+ str(perf_stat_runcount) )\n",
    "        # print ('Total Records: '+ str(record_counter) )\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Generating Combined dataset per test results"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 108,
   "metadata": {},
   "outputs": [],
   "source": [
    "def ProcessTestResults( data_perfstat_file:str,\n",
    "                        data_power_file:str,\n",
    "                        data_poll_file:str,\n",
    "                        outputcsvfile:str,      # Absolute/Full path to output CSV file\n",
    "                        tolerance_perf_to_power = pandas.Timedelta('50 milliseconds'),   # Tolerance value in UTC-time to merge perf-stat and SmartPower3 data\n",
    "                        tolerance_perf_to_poll  = pandas.Timedelta('5 seconds'),\n",
    "                        with_prof_results: bool = True\n",
    "                       ) -> None:\n",
    "    # Forming file names\n",
    "    # data_prof = os.path.join(resultsdir,testnameprefix+'.prof')\n",
    "    # data_power    = os.path.join(resultsdir,testnameprefix+'.powdata')\n",
    "    # data_polldata = os.path.join(resultsdir,testnameprefix+'.polldata')\n",
    "    if with_prof_results:\n",
    "        temp_data_prof_csv  = data_perfstat_file+'csv'\n",
    "\n",
    "        # Process the perf-stat file to generate intermediate CSV file for further processing\n",
    "        Process_ProfFile(data_perfstat_file, temp_data_prof_csv)\n",
    "\n",
    "        # Now read all the CSV files\n",
    "        prof_df = pandas.read_csv(temp_data_prof_csv)\n",
    "\n",
    "    power_df = pandas.read_csv(data_power_file)\n",
    "    poll_df = pandas.read_csv(data_poll_file)\n",
    "\n",
    "    # Drop unncessary columsn which are not required and not influencing current test scenarios\n",
    "    power_df.drop([\n",
    "                    'sm_mstime', 'localtime',\n",
    "                    'ps_ippwr-volts_mV', 'ps_ippwr-ampere_mA', 'ps_ippwr-watt_mW', 'ps_ippwr-status_b',\n",
    "                    'dev_ippwr-ch0-volts_mV', 'dev_ippwr-ch0-ampere_mA', 'dev_ippwr-ch0-watt_mW', 'dev_ippwr-ch0-status_b', 'dev_ippwr-ch0-interrupts',\n",
    "                    'dev_ippwr-ch1-status_b', 'dev_ippwr-ch1-interrupts',\n",
    "                    'crc8-2sc','crc8-xor'\n",
    "                    ],\n",
    "                axis=1, \n",
    "                inplace=True)\n",
    "    poll_df.drop( [\n",
    "                    'ts_local', \n",
    "                    'stat_cpuid', 'stat_user', 'stat_nice', 'stat_system', 'stat_idle', 'stat_iowait',\n",
    "                    'stat_irq', 'stat_softirq', 'stat_steal', 'stat_guest', 'stat_guest_nice'\n",
    "                ],\n",
    "                axis=1, \n",
    "                inplace=True)\n",
    "\n",
    "    # print ('Perf/Profile Data:')\n",
    "    # display(prof_df)\n",
    "\n",
    "    # print ('Power data:')\n",
    "    # display(power_df)\n",
    "\n",
    "    # print('Polled data')\n",
    "    # display(power_df)\n",
    "    if with_prof_results:\n",
    "        prof_df[\"utctime\"] = pandas.to_datetime(prof_df['utctime'])\n",
    "        prof_df.index = prof_df[\"utctime\"]\n",
    "\n",
    "    power_df[\"utctime\"] = pandas.to_datetime(power_df['utctime'])\n",
    "    power_df.index = power_df[\"utctime\"]\n",
    "\n",
    "    poll_df[\"ts_utc\"] = pandas.to_datetime(poll_df['ts_utc'])\n",
    "    poll_df.index = poll_df[\"ts_utc\"]\n",
    "\n",
    "    if with_prof_results:\n",
    "        df = pandas.merge_asof(left=prof_df, right=power_df,right_index=True,left_index=True,direction='nearest',tolerance=tolerance_perf_to_power)\n",
    "        df = pandas.merge_asof(left=df, right=poll_df,right_index=True,left_index=True,direction='nearest',tolerance=tolerance_perf_to_poll)\n",
    "    else:\n",
    "        df = pandas.merge_asof(left=power_df, right=poll_df,right_index=True,left_index=True,direction='nearest',tolerance=tolerance_perf_to_poll)\n",
    "\n",
    "\n",
    "    pandas.set_option('display.max_columns', None)\n",
    "    pandas.set_option('display.expand_frame_repr', False)\n",
    "\n",
    "    # print('Merged data')\n",
    "    # display(df)\n",
    "\n",
    "    print('Saving Merged data to CSV location: '+ outputcsvfile)\n",
    "    df.to_csv(outputcsvfile,header=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Main body to Iterate and process each results set"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Processing (1/2): /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_08-30-00_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.7GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.7GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.7GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_10-25-19_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.8GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.8GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.8GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_08-37-36_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.5GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.5GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.5GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-41-58_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.6GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.6GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.6GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_09-28-24_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.3GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.3GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.3GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_08-45-13_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.3GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.3GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.3GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-53-28_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.3GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.3GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.3GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_09-24-36_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.4GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.4GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.4GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-30-27_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.9GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.9GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.9GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-57-18_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.2GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.2GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.2GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_09-47-12_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.8GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.8GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.8GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_09-35-59_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.1GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.1GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.1GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_09-32-10_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.2GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.2GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.2GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_10-40-39_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.4GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.4GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.4GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_10-17-40_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-2.0GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-2.0GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-2.0GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-34-17_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.8GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.8GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.8GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_09-43-23_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.9GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.9GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.9GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_08-33-49_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.6GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.6GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.6GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-38-07_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.7GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.7GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.7GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_10-21-30_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.9GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.9GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.9GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-49-38_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.4GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.4GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.4GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-45-48_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.5GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.5GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-0.5GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_08-41-24_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.4GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.4GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.4GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_10-48-18_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.2GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.2GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.2GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_08-49-02_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.2GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.2GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-0.2GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-26-36_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.0GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.0GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.0GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_09-39-35_Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.0GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.0GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-LittleCore-60sidle-CPUFreq-1.0GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_10-36-49_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.5GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.5GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.5GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_10-44-28_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.3GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.3GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.3GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-14-2023_09-22-47_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.1GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.1GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.1GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_10-32-59_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.6GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.6GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.6GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/MaxFan/11-13-2023_10-29-09_Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.7GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.7GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/MaxFan/Idleworkload-MaxFan-BigCore-60sidle-CPUFreq-1.7GHz/Idling.powdata.csv\n",
      "Processing (2/2): /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_08-56-38_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.6GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.6GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.6GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_09-58-39_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.2GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.2GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.2GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_09-15-13_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.3GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.3GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.3GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_09-51-01_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.4GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.4GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.4GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-27-51_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.4GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.4GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.4GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_10-55-57_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.9GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.9GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.9GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_09-19-00_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.2GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.2GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.2GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-20-16_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.6GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.6GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.6GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_11-07-28_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.6GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.6GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.6GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_11-03-38_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.7GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.7GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.7GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-08-49_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.9GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.9GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.9GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_11-22-48_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.2GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.2GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.2GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-12-39_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.8GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.8GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.8GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_10-06-15_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.0GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.0GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.0GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_09-00-26_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.5GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.5GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.5GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_10-10-02_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.9GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.9GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.9GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-31-38_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.3GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.3GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.3GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_08-52-51_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.7GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.7GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.7GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_10-02-28_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.1GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.1GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.1GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-24-06_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.5GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.5GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.5GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_10-13-51_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.8GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.8GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.8GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-16-27_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.7GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.7GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.7GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_10-59-47_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.8GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.8GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.8GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_09-04-13_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.4GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.4GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-0.4GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_10-52-08_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-2.0GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-2.0GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-2.0GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_11-18-58_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.3GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.3GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.3GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_09-54-50_Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.3GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.3GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-LittleCore-60sidle-CPUFreq-1.3GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_11-11-18_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.5GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.5GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.5GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-04-59_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.0GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.0GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.0GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-01-07_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.1GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.1GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.1GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-14-2023_10-35-27_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.2GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.2GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-0.2GHz/Idling.powdata.csv\n",
      "Archive: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/results/01-Simple-Idling/NoFan/11-13-2023_11-15-08_Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.4GHz.tar.bz2\n",
      "Output to be placed in : /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.4GHz\n",
      "Number of results to process: 1\n",
      "Processing  File (1/1): Idling.powdata\n",
      "Saving Merged data to CSV location: /home/vaisakh/developer/modeling/E0240_ModSim-PowerMon/combined_dataset/01-Simple-Idling/NoFan/Idleworkload-NoFan-BigCore-60sidle-CPUFreq-1.4GHz/Idling.powdata.csv\n"
     ]
    }
   ],
   "source": [
    "# direcotories to process , so just converting them to absolute paths\n",
    "__results_archive_dirs__ = [\n",
    "        '01-Simple-Idling/MaxFan',\n",
    "        '01-Simple-Idling/NoFan',\n",
    "        # '02-Idling-PerfSleep/MaxFan',\n",
    "        # '02-Idling-PerfSleep/NoFan',\n",
    "        # '03-Workloads'\n",
    "\n",
    "    ]\n",
    "__results_archive_dirs_profflags__ = [\n",
    "        False,\n",
    "        False,\n",
    "        True,\n",
    "        True,\n",
    "        True,\n",
    "    ]\n",
    "\n",
    "\n",
    "__output_subdir__ = 'combined_dataset'\n",
    "__output_dirs__ = []\n",
    "regex_filename = re.compile(r'([\\d-]+)_([\\d-]+)_(.*GHz)\\.tar\\.bz2')\n",
    "results_archive_dirs = []\n",
    "for dir in __results_archive_dirs__:\n",
    "    results_archive_dirs.append(os.path.abspath('results/'+dir))\n",
    "    __output_dirs__.append(os.path.abspath(__output_subdir__+'/'+dir))\n",
    "\n",
    "# Now iteratively process the archives and create combined data set for each\n",
    "total_dirs = len(results_archive_dirs)\n",
    "curr_dir = 1\n",
    "for curr_archive_dir in results_archive_dirs:\n",
    "    print ('Processing ('+str(curr_dir)+'/'+str(total_dirs)+'): '+ curr_archive_dir)\n",
    "    output_dir = __output_dirs__[curr_dir-1]\n",
    "    if not os.path.exists(output_dir):\n",
    "        os.makedirs(output_dir)\n",
    "    \n",
    "\n",
    "    __res_archives = glob.glob(curr_archive_dir+'/*.tar.bz2')\n",
    "    for archives in __res_archives:\n",
    "        print ('Archive: '+ archives)\n",
    "        output_subdir = os.path.join(output_dir,regex_filename.match(os.path.basename(archives))[3])\n",
    "        print ('Output to be placed in : '+ output_subdir)\n",
    "        if not os.path.exists(output_subdir):\n",
    "            os.makedirs(output_subdir)\n",
    "\n",
    "\n",
    "        #Temporary files/directories for handling data\n",
    "        tmpdir = tempfile.TemporaryDirectory()\n",
    "        # print(tmpdir)\n",
    "        \n",
    "        data_dir = extract_data(archives,tmpdir)\n",
    "        # print ('Temporary extraction to : '+data_dir)\n",
    "\n",
    "        if (__results_archive_dirs_profflags__[curr_dir-1]):\n",
    "            __raw_results = glob.glob(data_dir+'/*.prof')\n",
    "        else:\n",
    "            __raw_results = glob.glob(data_dir+'/*.powdata')\n",
    "        filecount = len(__raw_results)\n",
    "        tctr = 1\n",
    "        print('Number of results to process: '+str(filecount))\n",
    "        for raw_result in __raw_results:\n",
    "            __prefix_ = os.path.splitext(raw_result)[0]\n",
    "            __basename__ = os.path.basename(raw_result)\n",
    "            __outfile__ = os.path.join(output_subdir,__basename__+'.csv')\n",
    "\n",
    "            pow_data_file = __prefix_+'.powdata'\n",
    "            poll_data_file = __prefix_+'.polldata'\n",
    "            # try:\n",
    "            print ('Processing  File ('+str(tctr)+'/'+str(filecount)+'): '+ __basename__)\n",
    "            # print (raw_result, pow_data_file, tctr)\n",
    "            ProcessTestResults(data_perfstat_file=raw_result,\n",
    "                            data_power_file=pow_data_file,\n",
    "                            data_poll_file=poll_data_file,\n",
    "                            outputcsvfile=__outfile__,\n",
    "                            with_prof_results=__results_archive_dirs_profflags__[curr_dir-1]\n",
    "                            )\n",
    "            # except ValueError as e:\n",
    "            #     print ('Invalid data entry found in one of the files for '+str(raw_result))\n",
    "            tctr += 1\n",
    "    curr_dir += 1\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Compress the combined_dataset\n",
    "source_dir = 'combined_dataset'\n",
    "with tarfile.open('combined_dataset.tar.bz2', \"w:bz2\") as tar:\n",
    "        tar.add(source_dir, arcname=os.path.basename(source_dir))\n",
    "#shutil.rmtree(os.path.abspath(__output_subdir__))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}