Final Wrap-Up

March 17, 2016.

Overall, this has been a really good experience for me. I really liked having a rat that is my own to take care of and train, and I really looked forward to doing it each day (I only skipped a few days I couldn't find time to train on). I've also said this before, but it really helped me learn the class material because I was using it in some form during each training session.

Remy was a pretty good learner, and the chart below summarizes her average number of responses over the course of the training sessions. The fact that she learned pretty well made it more fun; I'm sure i would have been frustrated to no end and not enjoyed the assignment nearly as much if she didn't. So, other than being frustrated sometimes, I can't think of anything that really needs improvement in the process.

What was most surprising to me was that the process was so difficult starting out (shaping, in particular). I guess I just thought it would be simple--I'm giving the rat food, and she should want food because I took it all away from her, so why would she do anything other than want that food? She made the choice often to not eat the food or to not work for it, and that was surprising to me.

Before this project, a misconception I had was that training seemed like something I wouldn't ever really be able to do. I tried to train my dog when we first got her almost 12 years ago, and none of that was at all successful or long-lasting. (Although, she is a pretty well-behaved dog anyway...) Now I feel like I will be able to use operant conditioning procedures in my daily life, if not just because I learned them but also because they are on my mind more frequently. I can really use this stuff!

To those who have been reading along with me as I've been doing this, thanks! It's been a lot of fun and I hope I've been providing entertainment and education!

-Jessica

Sniffy VS Remy

March 16, 2016.

I was fortunate enough to get to work with Sniffy the virtual rat and Remy the silly little Long-Evans rat for the first part of this course. Each had its pros and cons, and each taught me (haha) about operant learning!

For starters, shaping in Remy took 3 and a half hours of training in total, while Sniffy took about an hour to shape. I attribute these differences to the fact that Sniffy is a virtual rat that only has a certain number of programmed behaviors, and those DON'T include much of a curiosity or anxiety. Remy, on the other hand, spent a lot of time exploring and being anxious, which took away from time that we spent on training.

Magazine training in each of the rats was not terribly different, and Sniffy gave me a good basis for how exactly to train Remy. In each case I just offered reinforcement when the rat was close to or facing the food magazine, and they both finished magazine training in about 30 minutes without issue.

Shaping Remy, however, (as I alluded to earlier), was much harder than shaping Sniffy. Remy just wasn't all that interested in going near the bar or even on the same side of the box as the bar for a lot of the 3.5 hours we spent shaping. She was much more interested in the front corner of the box, and for a large portion of the shaping procedure, I couldn't do much about that. Sniffy just hung out next to the bar a lot and didn't spend time doing other things, so she got shaped faster.

The strength of using the Sniffy program comes from the fact that it is so simple to address the concepts of operant learning this way. I was able to practice without any problems magazine training, shaping, and schedules of reinforcement without something I had to hold or feed, and that just made it easier. Also, Sniffy was not prone to getting satiated from the food source, whereas Remy could have gotten bored or full and may have at several points in our training.

Here is a Remy's weight chart:

	Remy’s Weight Chart	Target Weight: 200.0 grams
Date	Weight (grams)	Food Given (grams)
2/17	222.0	2.5
2/18	210.0	4.4
2/19	205.7	5.7
2/20	204.5	4.3
2/21	199.5	9.9
2/22	207.0	6.2
2/23	203.0	5.6
2/24	199.8	6.1
2/25*	200.4	17.9
2/26
2/27*	198.2	17.7
2/28
2/29	196.0	7.4
3/1	198.3	8.0
3/2	198.9	7.6
3/3	199.5	9.1
3/4	199.1	9.5
3/5	198.1	10.0
3/6	201.9	9.1
3/7	197.2	10.3
3/8	201.5	9.7
3/9	200.5	9.4
3/10	199.5	10.0
3/11	203.6	8.6
3/12	202.4	8.9
3/13	198.6	10.4
3/14	198.0	10.8
3/15	200.0

*indicates days where Remy was fed the amount for two days to accommodate my absence

Of course, she ended the project exactly at her target weight!!

The biggest benefit of having Remy was learning patience along with the application of the 10 laws of shaping. She forced me to take the knowledge from class and from Sniffy and really use it, which was really cool. It was also really frustrating (see Troubles with Training post). I also enjoyed having an animal to take care of and train at a certain time every day because it really kept me accountable for the process.

Both virtual and live rats should be used in future classes. They are incredibly instrumental in showing us the concepts from class, making that material easy to learn. It was also fun to take care of a real rat and to be able to train a virtual one in any way I wanted to.

Kudos to Dr. Trench for the curriculum; I've really enjoyed it!

-Jessica

Remy's UNtraining--Extinction

March 13 and 15, 2016.

For the last two days of working with Remy, I put the bar pressing behavior on extinction. The purpose of this is so that we can hopefully see principles discussed in lecture in real life.  Extinction is a way to get rid of a response by no longer reinforcing it when it is present. Put in our terms, Remy will no longer receive a pellet for ANY of her bar presses. My goal would be for her bar pressing to decrease over time due to lack of reinforcements.

"You're going to do WHAT to me?!?"

The procedure for these two days was to set the computer program to extinction and then carefully watch Remy for changes in her behavior. Our training sessions began at 4:30 and Remy weighed in at 198 (89% of her starting weight).

Results:
On the first day, Remy pressed the bar 81 times, and she pressed it throughout the 30 minute session. On the second day, Remy pressed the bar 34 times, and all but 10 of those bar presses were in the first one-third of the session.

Cumulative Record 3/13/16 

Cumulative Record 3/15/16

I've also put together graphs that describe her number of responses per 5 minutes of each extinction session. Her responses clearly declined after 2 days of extinction but were not yet back to baseline, which meant that she could have used more sessions based on the definition of extinction.

Discussion:
Obviously, Remy was not a big fan of Extinction. During the two sessions, she spent a lot of time rearing (like trying to escape) and grooming (an anxiety behavior) when she wasn't attempting to press the bar. She also did some things I had never seen before, including biting the bar and scratching at the inside of the food magazine. Some of her emotional behaviors are below.

I did not see a spontaneous recovery but expect that I would have seen once potentially had I waited longer between the two extinction days (I only skipped one day in between) or if she had more days in the box that she was put on extinction.

The extinction process kinda made me feel bad for Remy after it was over, so I gave her a few more chocolate pellets and then put her on free-feed--which I'm sure she appreciated. Hopefully she has forgiven me!

-Jessica

The Troubles with Training

February and March 2016.

With how easy skilled trainers make it sound to use operant conditioning on an animal, I'm guessing a handful of my classmates walked into Psychology of Learning lab just like I did, thinking we were going to conquer the world AND train 22 rats with no problems.

Unfortunately, it didn't really work out like that at all. The rats were all kinds of problematic, and Remy showed some of these problems to me firsthand.

The main problem I had with Remy was when I went out of town over a 3-day weekend just after I believed her to be well on her way to being shaped. I was worried what this kind of time off would do to our training sessions, but I didn't expect her to basically regress back to her baseline of behavior. This was an incredibly frustrating experience because it meant several more days of patience before I really got anywhere. There was little I could do to fix it except keep on trying, and I don't think there was anything I could have done differently given I had no choice but to go out of town that weekend.

Another problem I had with training was that sometimes the box would malfunction when I set it to a protocol. As soon as I realized this had happened (it happened several times), I reset the protocol and began the 30-minute session over again. This didn't really seem to affect Remy, though I worried it would if she happened to press the bar while I was resetting, meaning she wouldn't get reinforced.

A final problem with training Remy was her overall anxiety level. There were definitely times I could point out that resulted in a lack or loss of progress due to the fact that she was often prone to jumping out of her skin for no reason. I think it was just her nature, though. I could have maybe managed this a little better with more handling time, or perhaps trained her when less people were coming in and out of the lab to train their own rats.

I think training went pretty well for the most part. There were little things I could have improved here and there such as more handling time and remaining calm/patient during sessions, but I think throughout the process I was patient and we were generally successful in how far we got.

Au revoir, Skinner box!

-Jessica

Sniffy's Schedules of Reinforcement and Extinction

March 14, 2016.

After magazine training Sniffy to associate the clicking sound with receiving food in the magazine and shaped her to associate a bar press with receiving food in the magazine, my goal was to put Sniffy on a schedule (or a couple) of reinforcement.

As you'll remember from my posts about Remy, the purpose of a reinforcement schedule is to solidify the behavior in the animal even if it is not being reinforced for every correct behavior. That is, Sniffy has to learn that she will not receive food every time she presses the bar. I wanted to do a couple of different ones for Sniffy with the goal of comparing the different types of schedules based on what we have learned in our lectures.

My procedure was to put Sniffy on a variable ratio schedule of reinforcement (VR5) and then a fixed interval schedule of reinforcement (FI30). I don't have to do much here except watch her press the bar and see how her patterns of behavior change. After allowing Sniffy to run on each of these for awhile, I put her on extinction.

Results:
The characteristic I looked for/expected on a variable ratio schedule was a steady rate of responses, theoretically because Sniffy would not know when to expect reinforcement with a given number of bar presses and would work hard at all times.

Cumulative record (in part) of Sniffy's VR5 schedule

For a fixed interval schedule, I would expect the cumulative record to look scalloped, indicating the increase of bar pressing near the end of the 30-second interval.

Cumulative record (in part) of Sniffy's FI30 schedule

Finally, I put Sniffy on extinction, which I'm sure the cyber rat would find quite disagreeable if she had feelings. She was reinforced for no bar presses, and it was expected that after an initial increase in the behavior, it would drop off dramatically. The process of extinction is exemplified and illustrated very well below.

Sniffy after ~30 minutes on extinction

Discussion:
Training Sniffy was a dream overall compared to training Remy. It took awhile to magazine train and shape her because the program has high thresholds for establishing that Sniffy has learned something (i.e., getting the bar operant associations bars close to their max levels) which does not really seem to be necessary in a live rat. On the other hand, all the variances of a live rat are not applicable to Sniffy, which makes the process seem shorter because it is less frustrating.

I spent approximately 45 minutes on each schedule of reinforcement and about 30 minutes on extinction. Magazine training took 45 minutes and shaping took even longer--an hour and a half. Throughout each of the different schedules, I didn't notice many oddities about her behavior except that on VR5 and FI30, her bar presses became more insistent and she spent more time near the bar even if she was doing something else such as grooming (similar to Remy on her schedules).

Sniffy the virtual rat is the ideal method to learn how the operant conditioning procedures and helped me a lot in the beginning stages of training Remy. I enjoyed getting to put her on several schedules of reinforcement because it gave me a chance to put the concepts from our class into practice.

Who knows--I may keep playing around with the Sniffy software after the class is over!

-Jessica

Remy's Training Days 15 and 16-VR schedules

March 11 and 12, 2016.

For these two days, I decided to put Remy on two variable ratio schedules of reinforcement. A variable ratio is different from a fixed ratio in that the number after VR(x) is the average number of responses required for a reinforcement to be issued. My goal for these two days was to challenge Remy, because it seemed like she was getting bored with the bar pressing game.

For these procedures, I just watched Remy press the bar. Which she did, a lot. Our sessions began at 4:30 each day and her weight was 203.6 grams on March 11 and 203.4 grams on March 12 (91% of her starting weight). Take a look below for an example of how she did:

Results:
I was hoping to see less post-reinforcement pauses in this type of schedule and an overall increased level of response in Remy. On the VR3 schedule, she pressed the bar 160 times and got 53 reinforcements. On the VR5 schedule, she pressed the bar 225 times and got 45 reinforcements.

Cumulative Record 3/11/16 (VR3)

Cumulative Record 3/12/16 (VR5)

As you can see, I still saw the pauses marked by a horizontal line, but there were less of them than on the fixed ratio schedules.

Discussion:
I would say that Remy was challenged by this type of schedule. Her behaviors changed during these training sessions, marked by higher levels of aggression towards pressing the bar, constant checking of the food magazine, and more time spent near the bar. She seemed confused often when it didn't deliver reinforcements, but wouldn't give up!

Now that I have tested her skills at variable schedules of reinforcement, I will spend my last two training days extinguishing the bar pressing behavior.

-Jessica

Remy's Training Day 12-14 (FR3)

March 8, 9, and 10, 2016.

These days of training, my purpose was to stretch the ratio that Remy could tolerate for not receiving reinforcements. The goal was for her to get at least 60 reinforcements on this schedule for a few days before I planned to stretch the ratio further.

My procedure for these sessions was essentially the same as the one for the FR2 sessions. To refresh your memory of the definition of an FR schedule, remember that FR3 means that Remy only gets a reinforcement every THIRD time she presses the lever. It is a fixed ratio (3:1) between the number of bar presses (3) and receiving food in the magazine (1). Our training sessions began at 4:30 PM each day. Remy weighed between 199.5 grams and 201.5 grams, maintaining roughly her target weight of 200 grams (90% of her body weight).

Results from these training days were mixed. The first day was great--Remy pressed the bar 180 times and got 59 reinforcements! For most of her time in the operant box, she was focused on getting pellets and little else concerned her besides the occasional grooming session. An example of her grooming is in the video below.

The next couple of days did not go so well. The second day, Remy spent the entire training session agitated. Just before the session, she tried to leap out of her cage and I stopped her, which unfortunately scared her half to death. I put her in the operant box just to watch her press the bar a total of 40 times in between her obsessive grooming sessions and attempts to climb out of the cage... :(

After the third day, I decided we would not spend another day on an FR schedule. This day, she pressed the bar 95 times and received 31 pellets, but I did not anticipate that she would progress any further or do as well as the first day of FR3 if she kept at it. Another note about this day that was different than other days: while Remy was focused enough to press the bar almost 100 times, she still seemed very agitated this day. I determined this by her behaviors--darting across the box, jumping towards the ceiling, and trying other alternatives to pressing the bar such as pulling it down from underneath her.

Here are the cumulative records for each day of FR3:

Cumulative Record 3/8/16

Cumulative Record 3/9/16

Cumulative Record 3/10/16

Discussion:

FR3 was the highest schedule I attained for the fixed ratio type of reinforcement. Remy's behavior changed a little from FR1-FR3, mostly via the assertiveness she had in pressing the bar when it didn't offer her reinforcements right away. Overall as I moved through the schedules, she spent more time at or near the bar and learned to press it several times in a row with persistence. 

Sadly, my training time with Remy is almost done. I have chosen to move on from a fixed ratio schedule because I feel like she has already shown me her best and now her bar pressing behavior seems to be declining. Because of this, the next post will cover my attempts at a variable ratio schedule of reinforcement (to be explained soon).

Until next time!

-Jessica