Extend GDB plugin to communicate with other plugins

[BlueHazzard]

i continued my work and have now finished the svd file parser.
My next step is to implement the api to read the periphery register from the microcontroller.

I thought a lot about this and i think the best solution would be to implement "memory watches". They would be implemented like the current watches but instead to use the "print" gdb command they would use "x" command.

Any thoughts about this?

On a second thought, i would like to save the window, where the watch is, into the "cbWatch" class (and also into the new cbMemoryWatch class). This would enable the ability to use the same memory watch for the ExamineMemory dialogue and the  periphery register dialogue. This would also allow to add multiple memory windows and watch windows like they exist in other IDEs.

[oBFusCATed]

Multiple watch windows aren't implemented because wxAUI doesn't support moving one pane to a random notebook, so there is no UI space left. Unfortunately I don't think wx will add this feature any time soon, so we'll have to probably think about some hacky solution.

For the memory watches - just add the appropriate flags in the GDBWatch class and do separate processing based on these flags.
I don't understand all the talk about the ExamineMemory dialogs...

Keep in mind that working on the current gdb plugin will most probably be a waste of your time...

Do you have some branch somewhere I can look at?

[BlueHazzard]

Multiple watch windows aren't implemented because wxAUI doesn't support moving one pane to a random notebook, so there is no UI space left. Unfortunately I don't think wx will add this feature any time soon, so we'll have to probably think about some hacky solution.

Multiple tabs for the watches window? Anyway, this is not part of this discussion, i wanted only show a second use case.


For future discussion: Keep in mind that i want to implement this on codeblocks sdk level, not GDB-plugin level, because the functionality is debugger agnostic. If the debugger supports memory read out (as the most do) the debugger supports svd/register readout

For the memory watches - just add the appropriate flags in the GDBWatch class and do separate processing based on these flags.

So you think about modifying the cbWatch class? This would be ok, what about to store the owner of the watch?

I don't understand all the talk about the ExamineMemory dialogs...

on embedded development it is quite handy

Keep in mind that working on the current gdb plugin will most probably be a waste of your time...

I don't think so:
1) We are talking about to replace the gdb plugin for years now, at the plugin is still the old
2) I need this functionality quite often...
3) As i mentioned at the top this is debugger agnostic

Do you have some branch somewhere I can look at?

i have started some work here: https://github.com/bluehazzard/codeblocks_sf on the new_cpu_register_window_01 branch
[Edit:] The svd parser is not pushed yet...

[oBFusCATed]

Let me know when you have everything published, so I can inspect it fully.

One note don't add non generic stuff to sdk classes. I can see you're adding some svd fields in the project. This is not a good idea. Add some generic storage and provide a way to define multiple parsers. As far as I can see svd is arm only and thus it is not generic.

For cbWatch, I don't know if I like the idea to be used in the cpu register ui. For sure I don't want to store pointers to windows in it. Probably you can add something like a purpose field or something like this. And definitely it is not good idea to use it in the examine memory window. This window must be made to work with arrays of bytes...

There a few subsystems of cb that are in worse state than the debugger, so they have priority in my todo at the moment.

[BlueHazzard]

One note don't add non generic stuff to sdk classes. I can see you're adding some svd fields in the project. This is not a good idea. Add some generic storage and provide a way to define multiple parsers. As far as I can see svd is arm only and thus it is not generic.

You know, exactly this is the point i wanted to state on the top of this discussion. This should not be part of the sdk or the debugger plugin, but you said me that it is ok to implement this kind of stuff into the sdk. I don't like this idea, so i will go one step back and try some other approach:

1) Extend the SDK with a class called cbMemoryWatch (pseudocode):

Code

class cbMemoryWatch 
{
  uint64_t m_address;  // start address of the memory watch
  uint64_t m_count;      // count of bytes
  vector<uint8_t> m_data; // The actual data

  bool IsChanged() const;
  const std::vector<uint8_t>& GetData()
}

This class will hold a watch on a continuous memory (RAM) area with m_count size. (so for example 256 Bytes form address 0x331E34 )

Extend the cbDebuggerPlugin class with this API functions:

Code

virtual cb::shared_ptr<cbMemoryWatch> AddMemoryWatch(const uint64_t address, const uint64_t count) = 0;
virtual void DeleteMemoryWatch(cb::shared_ptr<cbMemoryWatch> watch) = 0;
virtual bool HasMemoryWatch(cb::shared_ptr<cbMemoryWatch> watch) = 0;
virtual bool SetMemoryWatchValue(cb::shared_ptr<cbMemoryWatch> watch, const std::vector<uint8_t>& value) = 0;
virtual void UpdateMemoryWatch(cb::shared_ptr<cbMemoryWatch> watch) = 0;

the MemoryWatches will get updated like the normal watches on every hold of the debugger, but with the "x" gdb command and not with "print" like the watches.

With this API it should be possible to implement a SVD (or any other) view as a plugin.

[oBFusCATed]

I guess, I'm not getting the idea of svds.
In the beginning you wanted to plug them in the cpu register's window, now you want to plug them in the examine memory or make new window.
I'm totally confused.

p.s. m_count doesn't make sense when m_data is a vector...

[BlueHazzard]

i never wanted to put them in the cpu register window, that was your idea... Ok from the beginning:

1) Microcontroller have special function register (SFR), where the functionality of the periphery is controlled. This register are mostly stored in a RAM block, or at least accessible over a RAM instruction, so they can be read out with the debugger with the "x" command. They are called register, but have nothing to do with the CPU internal register like the EIP or the PC (program counter) found in the CPU.

2) SVD files are special files (from ARM) that describe the location and the value of this special function register. They describe that on location 0x123413 is the SFR called UART_CTRL and if this register has the value 0x44 then the UART is activated, but if it has the value 0x33 it is deactivated. The SVD file describes also how you create header files to control this registers and many more things. This files are provided by the hardware manufacturers of the microcontroller. SVD is some file standard mostly used for ARM, but similar files exist for other microcontroller

3) So I want a window (property grid tree) that displays me the status of this SFR. For this the debugger reads some part of the RAM and interprets the data in it with the help of the svd file

4) What is needed: A interface to the debugger that lets you read , not necessarily connected, parts of the RAM (1). Where can this be found: In the memory window, so my thoughts were to connect this two things.

The interpreting and displaying of the SFR is a microcontroller specific task, so i personally think it should be put in a plugin. The needed functionality to the debugger is only read pieces of memory, what every debugger should be possible, but the plugin needs a interface for this. My idea is to introduce the concept of "memory watches" as described above, so any plugin can add watches to the current running debugger and keep track of them with a shared_ptr (probably not the best way, but hey this is how it is made with the watches)

(1) for example 12 Bytes starting from 0xAA1234, 4 Bytes starting from 0xAA1223 ecc.

[oBFusCATed]

Are these things always read using the x command? For all controllers or this is stored in the svd?
The term memory watches is not a good one. It is close to data watches.
Is it possible for these things to have sub-fields? Or do they store just a single value?
Do we need to display them in the UI like we display a C/C++ struct?

The cbMemoryWatch proposal is not a good one because it expands the API too much.
Lets see if we can reuse the watches functionality for these things.

edit: What is the cheapest way I can get one such controller+jtag debugger for experimentation?

[BlueHazzard]

Are these things always read using the x command? For all controllers or this is stored in the svd?

As far as i can tell all big microcontroller map their periphery register to ram, so it should always be possible to read them from ram with x.

Is it possible for these things to have sub-fields? Or do they store just a single value?
Do we need to display them in the UI like we display a C/C++ struct?

I can not find any nice image but you can get a idea from here: http://www.forward.com.au/BluetoothLedDriver/PINB_PB2.JPG for AVR

PORTB is a periphery with the register PINB (at address 0x36 with the value 0x04), DDRB (at address 0x37 with the value 0x00) and PORTB  (at address 0x38 with the value 0x04)
Now if i would use the memory watch idea, i would create a watch with the address 0x36 and the count of 3. This would return me {0x04, 0x00, 0x04} and the plugin would interpret  it and fill the property grid..

This image is not ideal, because one register can be divided in more Bit fields (from different size) and every bit field can have multiple values, selected by a drop down list.  Each bitfield can have different attributes like: read only, write only, write only once ecc...
Also one register can be multiple bytes large (AVR 8Bit register vs ARM 32 or 64bit register), this is described in the svd file.

Lets see if we can reuse the watches functionality for these things.

i am open for ideas, but keep in mind that it should be fast and the data should be as raw as possible...

The cbMemoryWatch proposal is not a good one because it expands the API too much.

I can understand this point to some extend if there are a lot external plugins, but codeblocks, at the moment, is not a application with tons of external plugins (at least i am not aware of them), so breaking the API should not be a big deal.

edit: What is the cheapest way I can get one such controller+jtag debugger for experimentation?

probably a NUCLEO-F103RB, nucleo-f091rc or a other stm discovery board (ARM controller from STMicroelectronics ) with around 15€. The debugger (SWD) is included and you get the possibility to make a full j-link ( https://www.segger.com/products/debug-probes/j-link/models/other-j-links/st-link-on-board/ ) out of it. Note: i have yet to try to use the stm controller to work in codeblocks. At the moment i use an arduino due and a j-link from segger (Atmel ARM controller)

[oBFusCATed]

API Breakage is not problematic. I just don't want to make the API big. I prefer if we could re-use as much as possible.

Do you expect to add "watches" for all supported "registers" from the file at once or do you add them one by one (the ones you need at the moment)?

Can you find some example svd file and then show the UI we should generate? So we can have something more concrete example we could discuss.
If you read every register with a single x command then the performance will be rather bad (similarly bad as if you add many watches).
The commands are executed synchronously, if you want asynchronous execution then gdb/mi protocol should be used.

[BlueHazzard]

API Breakage is not problematic. I just don't want to make the API big. I prefer if we could re-use as much as possible.

But you have to give people a possibility to work with it.
With the current implementation of the watches, every watch saved in the interna of the debugger is mirrored to the watches window. For this purpose this is not wanted...

Do you expect to add "watches" for all supported "registers" from the file at once or do you add them one by one (the ones you need at the moment)?

Probably as needed. It would be nice to make the first read out to read all registers but this can be a lot data (11kByte) and so it is probably better to read only the current open registers. This should be as fast as possible...

Can you find some example svd file and then show the UI we should generate? So we can have something more concrete example we could discuss.
If you read every register with a single x command then the performance will be rather bad (similarly bad as if you add many watches).

I updated my test app to populate a property grid. This is by far not complete (1), but you will get a idea (img is attached)
The corresponding part of the svd file (only the uart/MR register is shown:

Code

   <name>UART</name>
      <version>6418H</version>
      <description>Universal Asynchronous Receiver Transmitter</description>
      <prependToName>UART_</prependToName>
      <baseAddress>0x400E0800</baseAddress>
      <addressBlock>
        <offset>0</offset>
        <size>0x128</size>
        <usage>registers</usage>
      </addressBlock>
      <interrupt>
        <name>UART</name>
        <value>8</value>
      </interrupt>
      <registers>
          <name>CR</name>
          <description>Control Register</description>
          <addressOffset>0x00000000</addressOffset>
          <size>32</size>
          <access>write-only</access>
          <fields>
            <field>
              <name>RSTRX</name>
              <description>Reset Receiver</description>
              <bitOffset>2</bitOffset>
              <bitWidth>1</bitWidth>
              <access>write-only</access>
            </field>
            <field>
              <name>RSTTX</name>
              <description>Reset Transmitter</description>
              <bitOffset>3</bitOffset>
              <bitWidth>1</bitWidth>
              <access>write-only</access>
            </field>
            <field>
              <name>RXEN</name>
              <description>Receiver Enable</description>
              <bitOffset>4</bitOffset>
              <bitWidth>1</bitWidth>
              <access>write-only</access>
            </field>
            <field>
              <name>RXDIS</name>
              <description>Receiver Disable</description>
              <bitOffset>5</bitOffset>
              <bitWidth>1</bitWidth>
              <access>write-only</access>
            </field>
            <field>
              <name>TXEN</name>
              <description>Transmitter Enable</description>
              <bitOffset>6</bitOffset>
              <bitWidth>1</bitWidth>
              <access>write-only</access>
            </field>
            <field>
              <name>TXDIS</name>
              <description>Transmitter Disable</description>
              <bitOffset>7</bitOffset>
              <bitWidth>1</bitWidth>
              <access>write-only</access>
            </field>
            <field>
              <name>RSTSTA</name>
              <description>Reset Status Bits</description>
              <bitOffset>8</bitOffset>
              <bitWidth>1</bitWidth>
              <access>write-only</access>
            </field>
          </fields>
        </register>
        <register>
          <name>MR</name>
          <description>Mode Register</description>
          <addressOffset>0x00000004</addressOffset>
          <size>32</size>
          <access>read-write</access>
          <resetValue>0x00000000</resetValue>
          <fields>
            <field>
              <name>PAR</name>
              <description>Parity Type</description>
              <bitOffset>9</bitOffset>
              <bitWidth>3</bitWidth>
              <access>read-write</access>
              <enumeratedValues>
                <enumeratedValue>
                  <name>EVEN</name>
                  <description>Even Parity</description>
                  <value>0x0</value>
                </enumeratedValue>
                <enumeratedValue>
                  <name>ODD</name>
                  <description>Odd Parity</description>
                  <value>0x1</value>
                </enumeratedValue>
                <enumeratedValue>
                  <name>SPACE</name>
                  <description>Space: parity forced to 0</description>
                  <value>0x2</value>
                </enumeratedValue>
                <enumeratedValue>
                  <name>MARK</name>
                  <description>Mark: parity forced to 1</description>
                  <value>0x3</value>
                </enumeratedValue>
                <enumeratedValue>
                  <name>NO</name>
                  <description>No Parity</description>
                  <value>0x4</value>
                </enumeratedValue>
              </enumeratedValues>
            </field>
            <field>
              <name>CHMODE</name>
              <description>Channel Mode</description>
              <bitOffset>14</bitOffset>
              <bitWidth>2</bitWidth>
              <access>read-write</access>
              <enumeratedValues>
                <enumeratedValue>
                  <name>NORMAL</name>
                  <description>Normal Mode</description>
                  <value>0x0</value>
                </enumeratedValue>
                <enumeratedValue>
                  <name>AUTOMATIC</name>
                  <description>Automatic Echo</description>
                  <value>0x1</value>
                </enumeratedValue>
                <enumeratedValue>
                  <name>LOCAL_LOOPBACK</name>
                  <description>Local Loopback</description>
                  <value>0x2</value>
                </enumeratedValue>
                <enumeratedValue>
                  <name>REMOTE_LOOPBACK</name>
                  <description>Remote Loopback</description>
                  <value>0x3</value>
                </enumeratedValue>
              </enumeratedValues>
            </field>
          </fields>
        </register>

(1) For example i would like to post the value of the current slected enumeration in the 3. column of the property gird, but was not able to do it... Also some icons would be nice. Make custom controls for only boolean register, display the bit field width in the 4. column and so on...

[oBFusCATed]

With the current implementation of the watches, every watch saved in the interna of the debugger is mirrored to the watches window. For this purpose this is not wanted...

Do you mean that there are GDBWatch and WatchesProperties objects? The cbWatch objects are managed by the debugger, because it knows best what additional data it needs for them to keep them up-to-date.

For this new cbWatch type we could add something like:

Code

cbWatch* cbDebuggerPlugin::AddPeripheralRegister(uint64_t address, uint64_t size, const wxString &id (or fullname) )

Probably as needed. It would be nice to make the first read out to read all registers but this can be a lot data (11kByte) and so it is probably better to read only the current open registers. This should be as fast as possible...

There is no way to be notified of changes in the memory locations (hw watch points are too limited for this purpose).
So if you want all "registers" to be updated at every "next line" or "step into" command then this will be massively slow.
In the watches window there is functionality that makes it possible to mark that you want only certain watches to be auto updated. And there is manual update command per watch. Similar mechanism should be use here, too.

[BlueHazzard]

Quote from: BlueHazzard on Yesterday at 02:19:38

    With the current implementation of the watches, every watch saved in the interna of the debugger is mirrored to the watches window. For this purpose this is not wanted...

Do you mean that there are GDBWatch and WatchesProperties objects? The cbWatch objects are managed by the debugger, because it knows best what additional data it needs for them to keep them up-to-date.

i looked again in the code and now i think i understand what is going on:
1) Watches window creates a new watch with Manager::Get()->GetDebuggerManager()->AddWatch() and stores the returned value as a shared pointer in a list
2) If the watches window needs to update its state it reads the data from the shared pointer what contains the actual data, updated by the debugger plugin.
so you use shared pointer to share data between the plugin and the interface

So your proposal should be ok.

For this new cbWatch type we could add something like:
Code: [Select]
cbWatch* cbDebuggerPlugin::AddPeripheralRegister(uint64_t address, uint64_t size, const wxString &id (or fullname) )

Again, i would not make this (the name AddPeripheralRegister) so special... Just lets read a count of bytes from some memory location. This would allow us to make this way more flexible...

There is no way to be notified of changes in the memory locations (hw watch points are too limited for this purpose).
So if you want all "registers" to be updated at every "next line" or "step into" command then this will be massively slow.
In the watches window there is functionality that makes it possible to mark that you want only certain watches to be auto updated. And there is manual update command per watch. Similar mechanism should be use here, too.

Let me describe the procedure:
1) Start the debugger
2) Open the "System peripheral window". This window lists all registers from the svd file in a property grid-tree. All nodes are closed.
3) The target is halted for the first time (on the main entry point, this is some default behavior if you are debugging in embedded systems. The debugger stops after a reset/start and whaits for the user to run the target)
4) Optional: All registers are read for after reset state ( i don't think this is needed)
5) The user expands one or more peripheral nodes from the "System peripheral window". The debugger updates this elements from the target
6) User hits "run"
7) Target runs until breakpoint
debugger halts and updates all expanded registers from the "System peripheral window"
9) The user changes one value from a register
10) The debugger writes the new value to memory
11) The debugger updates the register view (to check if the write was successful)
11) User hits run
12) Loop to 6)

[oBFusCATed]

Again, i would not make this (the name AddPeripheralRegister) so special... Just lets read a count of bytes from some memory location. This would allow us to make this way more flexible...

What do you mean by flexible here? I don't care much about the exact name, but I don't like MemoryWatch, because it is too similar to data breakpoint and tracepoints. Which are different things.

About step 1: How do you select the svd file? The user selects it every time the window is being opened? It is stored in a project/target/the window and the window resets it every time new debug session is started?
About Step 5: linking the expanding of the property and updating the values is not a good idea. At least for the watches it is a terrible idea, but I guess it might work for these fixed set of "registers" kind of window.
Step 8 will definitely be supper slow in the current implementation. I'm not sure if it will be fast in a gdb/mi implementation, too. The gdb/mi performance depends more on the gdb's performance than the integration.

[BlueHazzard]

What do you mean by flexible here? I don't care much about the exact name, but I don't like MemoryWatch, because it is too similar to data breakpoint and tracepoints. Which are different things.

It would be possible for other plugins to read out memory from the microcontroller and use it for own things (for example https://www.segger.com/products/debug-probes/j-link/technology/real-time-transfer/general-information/#tab-15668-1 this is only a general idea, there are countless other things, like generating memory maps from MMU and DMA and so on) In general, if you can read the ram from a microcontroller you can do a hell lot of things. With this kind of watch it would be possible to do all this things from a plugin.

About step 1: How do you select the svd file? The user selects it every time the window is being opened? It is stored in a project/target/the window and the window resets it every time new debug session is started?

Of course it should be saved at project or target level. But this can be done from the plugin and codeblocks provides a api for this, so no problems here ( as far as i know a plugin can add notebooks to the Project->Settings dialog, at least the gdb plugin does it and so there is also a place for the setting entry). The svd file is parsed on load of the project (and when the file is changed in the options of corse). The window is populated at the opening of the debugger, with all elements. The window provides a search control so the user can find the needed register pretty fast. The update of the data is then a other problem, but there is the possibility to optimize, for example find the best size between count of watches and size of a singe watch.
At the moment i would like to implement it and for this i need to know what api the devs would support. For optimization is later time, if the bottleneck is found.

About Step 5: linking the expanding of the property and updating the values is not a good idea. At least for the watches it is a terrible idea,

Why is this a bad idea? Only get the data from the debugger as you need it, because from my experience the problem is codeblocks<->gdb and not gdb or codeblocks at program level. I think the mi gdb plugin does this and it is fast and reliable. You can cache the values and only update if the debugger was running in the mean time... In any way it is faster then updating the whole register map (if the map is 11kByte big, on smaller targets it may be a possibility. Don't optimize until you know where the problem is...)