WEBVTT

1
00:00:00.000 --> 00:00:03.300
Another one of our tech talk sessions.

2
00:00:04.319 --> 00:00:22.680
I am going to close the video down and and start the PowerPoint presentation just want to do a check, let me know. You should chat session if you could, if you have any questions, let me know if the sound is coming through. Okay. And if you can see the presentation on your screen.

3
00:00:30.060 --> 00:00:48.570
Excellent, thank you very much again that the chat session. Those who have attended these these webinars before the chat session that's really important to us, we actually store those look at them later. I use some of the questions and answers into our FAQs. So please use that freely.

4
00:00:49.800 --> 00:01:14.190
And I am joined today by Sandoval and Donna and et from our texts in their conference CCE confer and also my co presenter is Karen Bell trauma from West dad, who's going to help talk through some of the data detailed issues and and topics. So good morning again.

5
00:01:15.240 --> 00:01:28.350
As you know, we run these webinars. I'm going to stop my video. Now if if that's good. So there goes. And so we run these webinars once a month, we try to cover various topics.

6
00:01:29.460 --> 00:01:34.650
That the student success metrics come up a lot because there's a lot of work on that.

7
00:01:35.790 --> 00:01:51.420
And it's, it's, it's the tool we're using to align across various initiatives. So as we do new releases we we bring these sessions back and give you an update on what's happening with the new releases so

8
00:01:54.210 --> 00:01:58.440
The I let me talk a little bit about the dashboard development process.

9
00:01:59.580 --> 00:02:06.900
As many of you know we had quite a bit of field engagement and creating the metrics we had a working group of

10
00:02:08.160 --> 00:02:12.360
Represented bodies of CIOs CEOs CSOs

11
00:02:13.410 --> 00:02:32.190
CEOs Dean's workforce non credit and so on, that got together and helped us for about a year. Here in figuring out which metrics made most sense and how we could simplify the number of metrics that we had before.

12
00:02:34.800 --> 00:02:42.990
And then we had a group of researchers about 13, I believe, who helped in making sure that the data was

13
00:02:44.130 --> 00:02:55.530
Structured correctly and the definitions were constructed correctly and so forth, significant effort from the success center. Dr. P group West dead and the results partnership.

14
00:02:58.650 --> 00:03:03.690
beta version of the site was released in November. It is available for you to look at

15
00:03:04.800 --> 00:03:10.740
At the end, you can search for student success center for the chancellor's office and you'll find it.

16
00:03:11.160 --> 00:03:27.870
And the link is also down below. You can get to it by going to data.cc CC o.edu or going to CC CC o.edu slash data either one of those work Google works as well. So lots of different ways to get to it.

17
00:03:29.010 --> 00:03:29.640
And

18
00:03:32.760 --> 00:03:33.360
So,

19
00:03:34.560 --> 00:03:40.260
As many of you know when we were trying to align these metrics we looked at the very top level definition.

20
00:03:41.520 --> 00:03:50.040
Of what the metrics were some, you know, for example of Guided Pathways was looking for leaders we dug a little bit into that. What did they mean who they were looking at

21
00:03:50.460 --> 00:04:01.440
And then we try to align those things, then when we release the first level, we got into more core details of what code was being used.

22
00:04:02.760 --> 00:04:09.000
For each portal and I know I'm down in the weeds but I know that we have researchers on the call. And I want to be really clear.

23
00:04:09.960 --> 00:04:22.170
That we can we can we can define the definitions that we are synchronizing everything based on completion, but then people's definition of what computers are very different.

24
00:04:24.330 --> 00:04:32.970
For example, you might think that they're completely are graduating person is someone who gets in an A or as or

25
00:04:34.770 --> 00:04:41.790
You know, and but somebody else's might be. They're thinking ADP. So they really want to just look at HTTPS doubts. That's the definition they used

26
00:04:42.330 --> 00:04:51.900
Some people might consider transfer and their mind they're thinking see issue or you see that's what they use to calculate various dashboards and then when we think of

27
00:04:53.130 --> 00:05:01.560
People who transferred that could be any transferred to any other institution four year institution. So those are where the definitional issues are

28
00:05:01.920 --> 00:05:11.880
I can. I wish I could tell you that all of those are resolved and and that's far from truth. I think we've continued to uncover issues, researchers have been very active and

29
00:05:12.450 --> 00:05:26.700
Were very engaged with us. So lots of inputs been coming in to continue to find that, and you see a lot of those changes and and refinements and additions and the new release that will go to production on Friday.

30
00:05:30.060 --> 00:05:39.180
For this release. So this is significant because we are adding the desegregation that we had promised to this level. So that's what we're going to walk through the actual

31
00:05:40.380 --> 00:05:42.450
Each metric and the desegregation to that.

32
00:05:44.220 --> 00:05:47.220
So we'll have another major release in May.

33
00:05:48.540 --> 00:05:53.310
Will be decided we'll be adding levels for district and region.

34
00:05:54.510 --> 00:06:01.770
Macro regions micro regions will be you'll be able to compare data between different areas.

35
00:06:02.460 --> 00:06:17.550
And we'll have the cohort technologies added right now. We do not have cohorts, which means I want to look at students that started at this period of time, and how are they, how is that cohort doing. We don't have that cohort yet.

36
00:06:19.260 --> 00:06:22.350
So that would be in May. We think that

37
00:06:23.520 --> 00:06:30.900
The may release will send out as file their commitments to have. We have made so far with what we will be releasing with this dashboard.

38
00:06:31.710 --> 00:06:45.840
There are lots of things we were thinking about doing there are lots of nice features that have been evolving in the launch port for the last seven years their links direct links to best practices that you can access

39
00:06:47.580 --> 00:06:58.890
So, so all of that will probably come as we progress with this dashboard, but for the basics of the data that we had promised we would deliver that may really should get us to that point.

40
00:07:01.530 --> 00:07:04.710
So getting help and support on these dashboards.

41
00:07:06.300 --> 00:07:15.300
If you have specific questions I you know I know the fields been sending them directly to me directly to us directly to our researchers and am is

42
00:07:16.560 --> 00:07:28.500
Directly to a RP or Western or RP group, I would only ask that if you could send those to research at CC CC. Oh, that edu that allows us to really

43
00:07:29.580 --> 00:07:40.650
Aggregate those questions and respond to them and then collect that and put it in our FAQ. So that would be the best place to send your, your questions.

44
00:07:42.420 --> 00:07:49.590
In terms of who is responding to those questions. I want to be clear that you know I wished we had a lot of resources at the chancellor's office.

45
00:07:50.010 --> 00:08:09.780
But, but that majority of our resources are actually coming to us through grants and contracts. So we're getting help from all the people I already mentioned to help respond to these resolve the issues and then get back to you with with answers. So, so just keep that in mind as well.

46
00:08:12.330 --> 00:08:23.460
Also on division Resource Center. We have lots of communities that are exchanging thoughts and ideas and answers and questions. So feel free to join those

47
00:08:24.090 --> 00:08:33.870
When you hear us talk about the dashboard team we're generally talking about the same group of people who are working on the student success metrics and the launch board.

48
00:08:35.220 --> 00:08:50.190
For various reasons. We want to make sure those two efforts are tightly aligned. And over time, fully integrated so there won't be this distinction such a distinction between the two. But for now, there is

49
00:08:50.640 --> 00:09:06.690
Seven years of implementation and refinements and launch board that we don't want to lose. So we'll get both of them to the same same level and platform as soon as we can. I'm going to pause for a second look at questions, if I may, so just bear with me a second.

50
00:09:11.670 --> 00:09:15.660
One more thing I should add in terms of the the authentication and login

51
00:09:16.980 --> 00:09:17.280
I

52
00:09:18.360 --> 00:09:23.190
The vision Resource Center requires a login authentication.

53
00:09:23.970 --> 00:09:35.040
The launch school board requires login authentication. We're working to see if it's possible to remove the login authentication out of launch board and just have all of this stuff aggregated

54
00:09:35.880 --> 00:09:47.490
We have discussed possibility of perhaps having two levels of authentication or one with no authentication and one with authentication for the vision Resource Center, but we haven't really gone

55
00:09:48.690 --> 00:09:58.650
Very far with that yet, but we just know that we're trying to resolve the visibility of these items so that people aren't getting trapped behind user ID password issues.

56
00:10:02.100 --> 00:10:03.510
All right. Um,

57
00:10:06.270 --> 00:10:14.730
The other thing that's happening is our researchers are working feverishly in the field to see if they can replicate that data the way that we are

58
00:10:15.420 --> 00:10:23.640
Creating it. So let me kind of walk you through the process with that a little bit. The data from from the from. Are you from the college's

59
00:10:24.030 --> 00:10:31.020
Goes to probably some sort of a post processing is my guess. And then it's sent to us. It goes into our MRIs.

60
00:10:31.860 --> 00:10:40.800
And then from the MLS we get an extract that sends it to the dashboard to the people who are building the dashboard, the student success metrics and launch board and those kinds of dashboards.

61
00:10:42.060 --> 00:10:52.080
Between the post processing. You're doing the the pre processing and posting that Mrs. Does the code that they send us in the data and the

62
00:10:52.710 --> 00:10:56.250
work that we do to get it into the dashboard. There are some

63
00:10:56.940 --> 00:11:05.100
Transformations that could occur. So we're really working hard with the with the researchers in the field to find out why.

64
00:11:05.460 --> 00:11:09.510
There are some discrepancies between the beta. They're calculating and beta that we have

65
00:11:09.930 --> 00:11:15.690
I do realize that some of the data we have. You don't even have, for example, wage data and those kinds of things you don't have so

66
00:11:16.200 --> 00:11:24.810
I don't think that the issues so much around those topics but but this data that you do have know that we're really interested in working with you.

67
00:11:25.590 --> 00:11:34.050
Probably our resources are limited right now, but I do know that we have been working with some of the researchers in the field to get resolve some of those issues.

68
00:11:34.950 --> 00:11:45.810
And certainly, I think once we delivered them a release will have a lot more time and and bandwidth to focus on further aligning and resolving the issues with all of you.

69
00:11:46.200 --> 00:11:53.580
But, but that doesn't mean that we don't want to hear about it. So please do raise the questions. Let us know where the discrepancies are

70
00:11:53.940 --> 00:12:05.970
I know that's probably the biggest concern and how you calculate the data to colleges as a match what some dashboard. So just, I just asked you to bear with us as we work through the issues together and

71
00:12:09.270 --> 00:12:16.950
Let us know what additional support you need. Sometimes we try to guess what the field needs, but please, it's it's really helpful to hear from you.

72
00:12:17.370 --> 00:12:31.470
If you're a researcher on the on the call know and i think i'll repeat this again later we will have a webinar with the researchers on February 4. I think that was posted on the RP listserv. So if you want to join that, feel free.

73
00:12:31.890 --> 00:12:46.980
We're going to really dive into the details of some of this stuff on on the fourth. So if you really want deep in the trenches discussions on the metrics, that's probably a good webinar jam okay may keep going.

74
00:12:50.310 --> 00:12:58.620
There we are conducting can regional workshops we call them connecting the dots integrated planning.

75
00:13:00.000 --> 00:13:10.290
It's basically to to using to use the student success metrics as the vehicle by which we are integrating and connecting all the different initiatives together.

76
00:13:10.800 --> 00:13:23.730
Focusing that around the students journey from recruitment, all the way to completion graduation and and job placement wage game. So I think we've already had really good

77
00:13:25.800 --> 00:13:36.900
Registration for those events, I highly encourage you to attend one of those with your leadership team that's where we're going to really talk about how all these different

78
00:13:37.320 --> 00:13:50.010
Data points and the initiatives tied together along with a student center funding formula. Guided Pathways, we will have experts from each of those areas joining us for these regional

79
00:13:50.550 --> 00:14:04.290
Workshops so highly encourage you to please attend those with your leadership team registered to attend them. Those are coming to a region near you. I will personally be there to

80
00:14:05.490 --> 00:14:13.290
Do overviews and and collect your feedback and your input. So hope to see it. One of those sessions.

81
00:14:14.700 --> 00:14:32.010
I think I already talked about the vision for success communities. So those are are good to join. They are user ID password protected so what you post is for that community doesn't get shared out to the public. So those are those are important communities to join.

82
00:14:33.270 --> 00:14:33.870
Again,

83
00:14:34.890 --> 00:14:45.150
Let us know what resources will help you in this work and know that our resources are limited, but we still want to hear. I want to know when when we do have the bandwidth and we do have

84
00:14:45.540 --> 00:15:01.020
Opportunities to add resources. We're actually responding to the resources you need that what we think you might be. So that's really important to keep that communication going, Okay, I'm going to pause for a second look at the questions, if I could, and then and then see if we can continue

85
00:15:17.370 --> 00:15:29.760
Some very specific to questions specific to data. So I'll hold on those yes this is this presentation is being recorded and will be archived

86
00:15:31.530 --> 00:15:40.170
If you don't know about the digital futures websites that is the website that will have all of our recordings are posted on

87
00:15:55.980 --> 00:16:03.660
So there seems to be some some technical issues that we weren't able to accommodate more than hundred participants for this workshop, so

88
00:16:04.200 --> 00:16:16.650
That is that it looks like I have 107 right now. So, but there might be some limitations that I don't know about will certainly I have technical people on the call. So they will be looking at that, I'm sure.

89
00:16:17.850 --> 00:16:26.760
All right, I'm gonna continue, if I could, um, new enhancements and revisions and this release, as I said, this is the release where the the

90
00:16:27.150 --> 00:16:41.910
long awaited did this aggregated data is being added additional metric on successful enrollment. This is really important that we're still sort of fine tuning that definition. And because it's a new metric for us. It is the way we

91
00:16:42.450 --> 00:16:45.810
Look at how many people applied versus how many people enrolled.

92
00:16:47.160 --> 00:16:53.070
in any capacity so so that's going to be an important one for us to more fun and refined as we move forward.

93
00:16:54.300 --> 00:17:05.610
Some, some revisions to metric calculations, as I said, this is where much of the input that you have provided us is helping us refine these metrics calculations.

94
00:17:06.660 --> 00:17:16.110
And so that that's a lot of that was included in this version. And obviously, one more year of data 1718 has been added.

95
00:17:17.250 --> 00:17:23.370
Mind tension with this. I'm going to be very open with you about this is to make sure that we get

96
00:17:23.730 --> 00:17:38.640
These dashboards and it's real time available to you as possible. I know none of you want to go and look at the dashboard. It's got data that's two years old. And so as we work through closing that time to delivery.

97
00:17:40.350 --> 00:17:43.380
We're getting into these hiccups where data may not be available.

98
00:17:44.280 --> 00:17:56.670
So, so just know that we will work through those issues. But the goal is to get this as current as possible to you and as calculated as accurately as possible.

99
00:17:57.480 --> 00:18:08.580
In terms of the materials that the link is right there on the presentation. The presentation itself the pdf of the presentation will also be posted after the workshop to this site.

100
00:18:10.500 --> 00:18:10.860
Okay.

101
00:18:12.510 --> 00:18:18.960
Here the desegregation categories. You can see gender, race, ethnicity, age group.

102
00:18:20.250 --> 00:18:42.510
LGBT Q category race car recipients and so on. As you can see, Perkins first generation foster youth disabled veterans and promise grant recipients. I want to go back to race and ethnicity. As you know, we recently expanded the race and ethnicity breakdown to nearly 200

103
00:18:44.250 --> 00:18:50.850
That was added to CCC apply so students will be able to report accordingly.

104
00:18:52.260 --> 00:18:58.290
The field will be added to MRIs and in August, I believe.

105
00:18:59.310 --> 00:19:10.830
So, once he said to me is, is when you'll start to see that breakdown in the dashboard. We don't have that all those breakdowns yet all the drill downs on race and ethnicity yet in the dashboard.

106
00:19:13.530 --> 00:19:25.680
Okay, I'm Karen, I don't if you can hear me. Please go ahead and connect in and unmute and see if you can help me walk through these

107
00:19:27.420 --> 00:19:28.770
That the actual metrics.

108
00:19:29.400 --> 00:19:30.120
Or steps.

109
00:19:33.840 --> 00:19:34.800
Let me take over, then right

110
00:19:35.430 --> 00:19:36.210
Yes, go ahead.

111
00:19:37.590 --> 00:19:44.280
Um, yeah. So as mead said we are showing with the drop down just aggregated results.

112
00:19:45.300 --> 00:19:53.010
What he has in the screen right now is attainment of the living wage, actually, that one's going to be recap. That one's been recalculate to align with a new

113
00:19:53.550 --> 00:20:02.550
Definition to the funding formula that just happened. So I don't believe that data is still being fed by that uncalculated data. So don't look at that right now.

114
00:20:04.080 --> 00:20:05.400
You want to go to the next slide.

115
00:20:07.650 --> 00:20:18.420
So when there's compound metrics like transfer math and English. If you're familiar with the current dashboard. The 1.0 release is what we tend to refer to it as are the beta

116
00:20:19.230 --> 00:20:27.900
Right now, you would only see trasylol math and English together. So for those we're we're showing you each category so you can see what the rate is for

117
00:20:29.070 --> 00:20:37.440
Math and English. Remember with this is within the district in the first year to align with the funding formula, but then you can also see for a bath only in English only

118
00:20:38.460 --> 00:20:39.840
And. Next slide please.

119
00:20:45.330 --> 00:20:51.630
So yes, these are the list of additional metrics that we've added from the view that you can have seen now in the one point O

120
00:20:52.380 --> 00:21:02.340
Is Amin said successful enrollment that CCC apply data retained fall to spring it any Community College. So not just in the district that any Community College.

121
00:21:03.030 --> 00:21:19.050
As I just said before, we're showing much as a little math completion transferable English completion. So you can kind of break down that composite metric of transfer level math and English. And we also have the vision goal definition of completion and our success section.

122
00:21:20.190 --> 00:21:36.180
In addition, we have average number of units accumulated by a as students as well as 80 students and an average number of units accumulated by all associated to be learners. So those are, those will be added. Those are the new ones that you'll see when the dashboard goes live

123
00:21:37.980 --> 00:21:47.220
And then so there were some method methodological changes in the second release, quite a few. A lot of them based on your wonderful input in questions and

124
00:21:47.520 --> 00:21:52.170
Also just we're continually working with me is to understand how they've constructed metrics.

125
00:21:53.160 --> 00:22:02.370
That you see on the on the data mart as well as students that are funding formula. So it's all a little bit of a moving process as we as we work on this to try to clarify.

126
00:22:02.760 --> 00:22:09.930
And we do really appreciate your questions. I think I'll means made that abundantly clear, but I just would like to say how much we appreciate your feedback and

127
00:22:10.890 --> 00:22:18.300
Questions understanding that is quite frustrating to see these different numbers out there, but that we are so committed to working with you to figure out why they are different.

128
00:22:19.200 --> 00:22:38.280
Karen. If I could just add there as well. When you send us suggestions and corrective actions and issues that you uncover and we respond to those we we correct. Obviously we correct the data element dictionary be correct and metrics, the calculations. Just know that

129
00:22:40.170 --> 00:22:47.790
The person who reported it as excited about that people who didn't report it get confused. Right. Why didn't the data element dictionary change.

130
00:22:48.180 --> 00:22:52.290
Just know we're trying to correct this stuff. Now, obviously there is

131
00:22:52.740 --> 00:22:59.790
A strategy to say we're not going to change the DB, we're not going to change anything until you know every six months or something.

132
00:23:00.060 --> 00:23:06.900
But, but I don't think we're in that stage yet. I think we're in the stage of making sure everything is correct, as you look at this stuff and

133
00:23:07.140 --> 00:23:16.770
resolve any issues as quickly as we can. So you're going to see some morphing obviously the data and dictionary in the data in the dashboards. As we continue to refine

134
00:23:17.010 --> 00:23:30.360
And we want to be responsive to your issues that you're identifying so so it's it's a bit of a good and a bad there. And I understand. So the strategy. We've taken is what I described, but if you as a field.

135
00:23:30.990 --> 00:23:42.450
in totality think that is better, we wait for all the changes for, you know, six months releases or something. We can certainly do that. And that may be something we try after the may release. But, but let us know.

136
00:23:46.830 --> 00:23:56.400
Um, so this. So basically, just a reminder that it's just like the first the first dashboard that's up now is all a student journey based metrics.

137
00:23:56.820 --> 00:24:03.330
So this is really trying to understand how different populations of our students are doing. That's why there's some

138
00:24:03.810 --> 00:24:10.770
Outcomes for some journeys and others others that don't show up because they don't make any sense based on the student journey.

139
00:24:11.250 --> 00:24:18.630
And then, of course, I know a lot of people are interested in the all student journey which is of course the all students that that combine all journeys and of course they don't

140
00:24:19.140 --> 00:24:26.430
They will never total up exactly that's that's in the De de the exhibition in there, why I'm and then go ahead. Next slide.

141
00:24:28.470 --> 00:24:34.500
And so here, this is just a reminder of who is in the student journeys. We've made a few tweaks since the

142
00:24:35.010 --> 00:24:45.240
First version with a second version that's explicitly explain really well in the FAQ that Kathy's put together a Kathy booth is put together an abundance of resources to help explain any differences.

143
00:24:46.260 --> 00:24:56.070
So adult at ESL short term career degree transfer and then undecided. Other. So those are the four journeys, they're staying basically the same.

144
00:24:57.210 --> 00:25:04.740
Between two. The two versions, besides a few tweaks, but you'll see the same kind of visualizations dashboard and. Next slide please.

145
00:25:05.580 --> 00:25:10.380
So again, this is focusing on sort of progression, when you look through the metrics. The logic is

146
00:25:10.830 --> 00:25:27.450
Enrollment progress momentum success, implement and earnings. It's a little bit like the pipeline in the terms of the way the views are displayed in a sequence I sort of logical sequence of a student journey and. Next slide please.

147
00:25:28.080 --> 00:25:29.880
Before we get to the next slide.

148
00:25:30.960 --> 00:25:33.930
I just wanted I'm looking at the questions as they're coming in.

149
00:25:33.930 --> 00:25:34.380
And

150
00:25:34.470 --> 00:25:37.860
If it's a critical one. I'll just throw them. So we don't forget

151
00:25:39.390 --> 00:25:45.360
In terms of equity. I just want you to know that we we want to be as responsive as we can to the to the

152
00:25:45.780 --> 00:25:56.820
equity plan that you are required to submit and work on. So we're actually working on a spreadsheet or CSV file that we could publish to all of you, and especially the researchers

153
00:25:57.240 --> 00:26:06.570
That pre calculus all this stuff that you need for the the equity metrics that you need, because we do realize that what we have on the dashboard doesn't quite

154
00:26:08.370 --> 00:26:19.140
meet all the needs. So we're going to give you a spreadsheet that that captures all of that and pre calculate it for you. So just stay tuned, we're hoping to have that to you within a week or so.

155
00:26:21.180 --> 00:26:22.380
So we can go ahead please.

156
00:26:22.440 --> 00:26:23.010
Okay, great.

157
00:26:24.810 --> 00:26:35.490
Um, so I'm just gonna, we're gonna go kind of step by step for these different views that they're on the dashboard that I'm the only one. Again, that really knew is that a successful enrollment. The other ones are already there.

158
00:26:36.150 --> 00:26:50.850
So you want to go ahead. Um, so it does apply to all for student types. It tells you applicants enrolling the same college or really enroll. It's not in a different college. It's really in any community college. And the next slide please.

159
00:26:54.480 --> 00:27:04.050
So then learning progress to be the next step that is currently in the dashboard and that applies to only adult ed undecided. Other end all

160
00:27:06.030 --> 00:27:07.920
And so you will see skill gains and then

161
00:27:09.120 --> 00:27:15.450
That's very similar to what's being shown currently on the dashboard. And the next slide please.

162
00:27:18.750 --> 00:27:30.150
Of course success rate that is another metric that is currently on the dashboard and it applies to the short term career. I decided and other there was an adjustment methodology.

163
00:27:30.870 --> 00:27:40.860
We're aligning to the core success definition with data mart. So that's a change from the first version. And again, there are, there will be

164
00:27:41.700 --> 00:27:53.850
Lots of documents posted if they're not already posted on a little bit not up to speed in terms of what's posted right now and digital futures. But there's will be all kinds of explanations in terms of what has changed between the two views.

165
00:27:55.020 --> 00:27:58.890
And the De de we've really I believe have finally gotten to a place where

166
00:27:59.220 --> 00:28:05.040
The d g is. And of course it's not perfect, but I felt could we ever got the DVD perfectly wouldn't get anything up.

167
00:28:05.310 --> 00:28:15.810
But, and I am I am pretty much the one responsible for the DVD and I do appreciate the feedback that we've gotten to a place I think we're try really hard to break down how we've defined a metric

168
00:28:16.710 --> 00:28:19.290
Into a numerator and it doesn't matter if that makes sense.

169
00:28:19.680 --> 00:28:30.570
And each part of the calculation to be really clear what data elements we're pulling from so that when you're there trying to replicate the data you have the best information possible but again.

170
00:28:30.840 --> 00:28:37.080
If you're looking at the DVD that will be released with a site and you see things that are not still not clear, please let us know.

171
00:28:38.130 --> 00:28:38.400
And

172
00:28:39.360 --> 00:28:49.110
Also, when you look at what these metrics. The team has done everything they can't to figure out, you know, what additional data can we provide here that would help.

173
00:28:49.680 --> 00:28:59.520
The fields, for example, what's the source of the data is down the bottom of, you know, what's the numerator and denominator for this calculation and what is the description of it up on top.

174
00:29:00.300 --> 00:29:12.480
Let us know if this is good or bad or if there are additional information that would be good to have immediately when someone looks at the metric without having to go and search for it. So we're very open to that.

175
00:29:13.980 --> 00:29:23.010
The question was asked, and whether we can all the source data will be available for for colleges and others to replicate this calculation.

176
00:29:23.430 --> 00:29:41.670
And I don't know if that necessarily would be because a lot of this data is you know obviously PII data that's coming in per students being calculated. So I don't know if we will ever make all the data that's in the dashboard available for people to recreate the metrics.

177
00:29:42.960 --> 00:29:52.590
But perhaps we could talk a little bit more about details. So when someone asks, would the source be available. What, what exactly they mean by that, sorry, Karen. Go ahead, please.

178
00:29:52.650 --> 00:30:00.690
Oh no, yeah, no, of course. Um, and then still learning progress and this applies to a degree transfer and aside other and all

179
00:30:01.440 --> 00:30:07.620
It can be looked at this metric for us that completed translator math and English. And you'll see it broken down for just math and just English and

180
00:30:08.010 --> 00:30:12.990
Yeah, it is his own. He pointed out, so this is very similar to how the dashboard looks now but

181
00:30:13.410 --> 00:30:19.950
I always like under the title I do this myself to just be reminded, exactly what this metric is which is a longer description.

182
00:30:20.220 --> 00:30:31.530
Which is a gray among all students proportion you completed trestle math and English and their first academic year of credit enrollment within the district. We try really hard in a limited number of characters to describe

183
00:30:32.160 --> 00:30:38.400
What the metric is so I always sort of go there to remind myself. Okay. Wait, this is yes within district.

184
00:30:39.750 --> 00:30:47.280
First, and I know that saying that there's still a lot that could be could be defined different ways. And that's what you'll see in the DVD how we defined it.

185
00:30:47.880 --> 00:31:02.040
But please give us feedback on anything like that. That would help with the clarity of that. And again, there's a new methodology for this is adjusted to align to the core success definition with day tomorrow. So, um, it does now show math and English. And the next slide.

186
00:31:03.150 --> 00:31:08.040
Just in addition to this, there's been a lot of questions about when you talk about academic

187
00:31:08.340 --> 00:31:22.710
mackerras to the academic year for the college or is it the first year for each student, which means it could start with spring and then go into summer and fall I know those questions are coming in. We haven't tackled that yet. Just know that we've heard it, but we haven't tackled it

188
00:31:24.930 --> 00:31:25.860
Okay, great.

189
00:31:28.410 --> 00:31:39.030
And. Next slide. And so then the next section would be momentum. And so one of the metrics and is completed a level of education. Now this metric has

190
00:31:40.170 --> 00:31:47.820
Changed a lot from the first dashboard, I believe that we did not have and I'm trying to get confirmation from our data scientists coders.

191
00:31:48.180 --> 00:31:55.260
But you're gonna see a big change in this one. If you're, if you're paying attention to this one in the prior dashboard and we'll have some language out there to try to

192
00:31:55.710 --> 00:32:00.930
Say what was happening, it's, it's a bit more clear because there's some big decreases here in the in, especially in the numerator.

193
00:32:02.070 --> 00:32:11.970
And we're ensuring that the the populace for capturing this metric are only a V for ESL or ASC actually students who would have

194
00:32:12.870 --> 00:32:15.390
Completed a level education so that we might have gotten

195
00:32:16.020 --> 00:32:21.900
it wrong. We might have had the coding a little bit wrong in the first dash by the legit really high percentages. So

196
00:32:22.200 --> 00:32:33.840
Just so you know, we're going to work on some language that will come out that will explain the difference. We got a whole document explains the differences between version one and two for every metric that had a significant change and. Next slide please.

197
00:32:34.650 --> 00:32:42.630
Sorry, before I go, just two questions came in, I want to respond to those in terms of math and English.

198
00:32:43.440 --> 00:32:50.100
I think the question was, what, you know, Rican I rely solely on the top codes that are assigned to those courses now.

199
00:32:50.520 --> 00:32:58.620
I've watched to know that there has been an effort that Laura hope and I kicked off a little while ago and Kathy booth has

200
00:32:59.010 --> 00:33:15.210
has facilitated the project management of an effort. What can we, the academic senate and us and everybody else to try to figure out how we can better code the math and English courses so that we can make sure we capture

201
00:33:16.560 --> 00:33:22.680
That the types of courses that are either acceptable for transferable math and English by the CSU and you see

202
00:33:23.130 --> 00:33:30.300
And and that those are captured in our coding practices. So know that that proposal is coming through. I think

203
00:33:31.020 --> 00:33:39.660
academic centers going to take us through this pipeline of getting that word out to the colleges will certainly once we get the green light will share that openly with everyone.

204
00:33:39.990 --> 00:33:49.110
But yes, that effort is going on to better identify transfer level math and English courses. The other question that came up was

205
00:33:50.460 --> 00:34:06.330
Wait on me. Did you change the date from from January 31 for the release to February one, and I have to shamefully say yes. I think we will going to release it on Thursday, but it looks like they got a trailer when he said on Friday.

206
00:34:08.250 --> 00:34:15.270
So, so I apologize for that, that, that one day delay on there but yes it is being moved to Friday.

207
00:34:16.950 --> 00:34:19.980
I'm sorry, can please keep going. I'm going to change this slide.

208
00:34:20.640 --> 00:34:27.360
Okay. So, continuing in the momentum section completed a non credit work first milestone, there was a change in the methodology.

209
00:34:28.770 --> 00:34:31.740
And this this metric does exist in the current dashboard.

210
00:34:33.060 --> 00:34:41.280
And a new criteria. That's right. We have new criteria, it does now include students who completed her head 48 plus contact hours and non credit workforce preparation course.

211
00:34:41.730 --> 00:34:52.140
So again, we're going to have explanation in a document that explains exactly what those changes for and again for the for the exact calculation, please refer to the DVD and. Next slide.

212
00:34:59.160 --> 00:35:08.730
So, or nine plus career education units. We do have an explanation and the FAQ, I believe document. I'm not sure it's posted quite yet. I just saw this morning from Kathy.

213
00:35:09.030 --> 00:35:14.250
I saw there was a question about this metric in the chat box and why it's so low.

214
00:35:15.240 --> 00:35:31.380
So again, we did adjust the course success alignment day tomorrow and enrollment for the first time in a district and CTE course definitions to align with the Student Success funding formula. So, um, this so, so I think believe in the chat was asking about why this looks low

215
00:35:32.820 --> 00:35:41.640
Compared to right here. I'm seeing write a short term care, education, you've earned an endless looks low right so

216
00:35:42.660 --> 00:35:51.600
Again, please join the February 4 webinar. This is when we'll go into more details about these kind of issues. I'm trying to just kind of get through the overview of this new release.

217
00:35:52.770 --> 00:35:56.220
And since we don't have the time to delve into anything please join that but just know that there is

218
00:35:56.970 --> 00:36:10.710
Most your questions. I see so far. They're related to any of the data questions are will will have answers and we're docs or please join the webinar on the fourth to get more into those details and. Next slide please.

219
00:36:13.560 --> 00:36:19.890
Okay successfully lead at thresholds in the fall, we redefine these thresholds to align with data mart. We have a little bit different.

220
00:36:21.510 --> 00:36:22.110
And

221
00:36:23.220 --> 00:36:29.580
So this is actually students, then who completed real clickable unit thresholds in the fall term of the selected year

222
00:36:30.780 --> 00:36:40.440
And I THE HIGH PERCENTAGE OF COURSE OF THE FIRST categories zero would be due to students who did not pass the course. So if they got lower than

223
00:36:41.040 --> 00:36:54.660
I believe it's a D or because we did have a very liberal definition of students to be basically any enrollment. And so this is why we're seeing that heavy weight of students in that category and. Next slide please.

224
00:36:55.680 --> 00:37:08.460
So completed unit thresholds integrity year this is similar metric, but not in the fall. It's in the academic year for the different buckets. And I believe some of these buckets and align with what's needed for the vision. Vision for success goals.

225
00:37:09.540 --> 00:37:11.310
And. Next slide please.

226
00:37:12.420 --> 00:37:15.000
This is the new metric retain from fall to spring.

227
00:37:16.170 --> 00:37:23.520
I believe we have as much of a new methodology. So exclude students who transferred to any post secondary institution we learned from si M is that

228
00:37:24.120 --> 00:37:32.460
When they when they are excluding students who transferred. They do include students who transfer to any post secondary, not just for years.

229
00:37:33.420 --> 00:37:47.400
So that was a bit of an aha moment for us. And then there is an additional column. This is what's new in this dashboard. The additional column shows students who were chained anywhere in the system, not just at the same college

230
00:37:48.540 --> 00:37:50.010
Okay, next slide please.

231
00:37:50.340 --> 00:38:00.150
Just want to add that we are seeing at the state level that students to jump around quite a bit. I think you know you all know that. But so it seems like they abandoned us

232
00:38:00.540 --> 00:38:07.920
If we look at just one college but if in fact they haven't gone to a different color. So this was an important add on for us to capture

233
00:38:12.090 --> 00:38:13.290
And. Next slide please.

234
00:38:17.100 --> 00:38:27.930
Okay, so now we're in the success portion of the dashboard and this is where a lot of feedback we've had quite a bit of challenges getting these categories. Right. Um,

235
00:38:28.560 --> 00:38:38.490
And I do believe, hopefully now thanks to a lot of your feedback we've gotten them pretty pretty well nailed down in terms of transition to post secondary coursework.

236
00:38:39.030 --> 00:38:54.150
Is the first cat at earn a non credit certificate or Chancellor's Office approve certificate or an associate degree or an associate degree for transfer a change apprenticeship journey status. I don't believe we had that data in the first the

237
00:38:55.200 --> 00:39:07.950
First Release. We didn't November. What we do now have that data and then attain the vision gold completion. So that's basically how the vision, goals, define what success is and that's pretty clearly spelled out in the DVD.

238
00:39:09.030 --> 00:39:19.080
And again, there was a little bit of new methodology for the transition to post secondary. Again, we spent all that and documents. I'm translating to a four year. Sorry, go ahead of me to the next one.

239
00:39:20.220 --> 00:39:21.870
So transfer to a four year

240
00:39:22.980 --> 00:39:32.190
This data actually we didn't have the full transfer data when this slide was produced, I believe. So these numbers will be changing when you bought a dashboard. If you were to look at this

241
00:39:33.360 --> 00:39:46.500
So we do are breaking out by transfer to a CSU or you see or transfer to any out of state institution or transferred to a private college and state and of course the only source we have for anything that's out of state is a National Student clearing house.

242
00:39:48.150 --> 00:39:59.370
And so this is an average number of units accumulated by associates returners. We do have a calculation calculation error was a weighted average error that we just discovered

243
00:40:00.690 --> 00:40:12.540
With this metric. So that shows the first the first bar that has 92 units for all degree earners, which is lower than the two other buckets of a and as TV earners and

244
00:40:13.140 --> 00:40:18.510
At earners that that weighted average way was being calculated was not caught in our testing.

245
00:40:18.930 --> 00:40:32.130
So that has been fixed. And by the time this is released on I guess it's Friday. Now, Thursday, Friday, and you will see that be a different number. So, it will be for sense that that would not be lower than a as an agency years

246
00:40:33.060 --> 00:40:35.910
If I can add, sorry. Go ahead. Can

247
00:40:36.540 --> 00:40:40.770
I just want to say this is a big is a big new methodology that was discovered quite early.

248
00:40:42.960 --> 00:40:53.070
early winter the threshold for inclusion. So to shifted from 60 units at a college to 60 units in the system to align with the vision, goals, pretty important.

249
00:40:55.170 --> 00:41:01.740
I just wanted to add that, yo, yo, a here. Karen and I point out some of the areas that we have found

250
00:41:02.670 --> 00:41:16.140
And I hope we're not giving the impression that there are a zillion areas and this their, their zillion calculations that were done so for pointing out the few that we have uncovered this really only to respond to the ones that you have identified so

251
00:41:17.640 --> 00:41:32.880
So it's it's really in in scale factor, we're talking about, you know, compile handfuls versus the literally thousands of calculations and formulas that are driving this dashboard. So I just hope we're not giving the incorrect perception there.

252
00:41:34.050 --> 00:41:38.160
Right, yeah. And when these actually this this screenshots were done, which is quite a few days ago.

253
00:41:38.850 --> 00:41:51.300
That that dashboard has changed quite a bit. Even now on the staging site. So if I was, if, if possible, I would have updated like the most current screenshots from today and that that would have changed this one slide.

254
00:41:52.170 --> 00:42:09.420
Okay. Next, please. And employment. So again, I know this is the one only place really to get this kind of information. So among all students is proportion of students who are unemployed and became employed. So this is only applied to adult add ESL undecided and all

255
00:42:10.770 --> 00:42:11.550
And. Next slide.

256
00:42:13.170 --> 00:42:23.490
And job closely related to fill the study. This is the only metric coming from the CTE outcomes survey, since in our add your wage file, we cannot match.

257
00:42:24.240 --> 00:42:39.300
We do not know what occupation is student is employed in we get that information from self reported data. So it is important. This is survey data and it's just like, it's basically just like the metric that we've always seen on the CC pipeline.

258
00:42:40.350 --> 00:42:50.970
In terms of how trying to determine if our students are working in that job thats related to how what they studied in school and. Next slide please. So finally earnings

259
00:42:52.170 --> 00:43:08.880
So this median annual earnings is actually this is one that was a last minute decision with a calculation of the student centered funding formula. So we will get that be in the information the dashboard. When this slide was printed again is another one of those where

260
00:43:10.170 --> 00:43:20.670
It the the calculation actually was still being redone to align with the student centered funding formula. So that is something that will, you will see different when you go to the live dashboard on on Friday.

261
00:43:21.990 --> 00:43:31.890
That will be changing. And we will tell you exactly what actually counties have the new methodology here. It takes earnings and second fiscal quarter after exit and multiplies by four.

262
00:43:32.820 --> 00:43:41.010
Instead of taking all four quarters. And so that was to align to how they did the Student Success funding formula and. Next slide please.

263
00:43:41.640 --> 00:43:50.040
If I could just add on that. So that's the same formula. By the way, they get used this new methodology is what gets us to a handful federal

264
00:43:51.270 --> 00:43:58.740
Wage calculations. So, so we're really just aligning it with with other metrics and and methodologies that are out there.

265
00:44:02.250 --> 00:44:09.270
And then the medium change and earnings, this, this one is locked down, it won't change. Um, so it did have a little bit of a methodology.

266
00:44:10.830 --> 00:44:18.030
And so it does, we realized that we need to feel a little bit more specific about data entry for students. Someone pointed out that

267
00:44:18.660 --> 00:44:26.490
If they had taken a course as a young person and that was their first entry into the system. We might have been grabbing that wage from that long ago.

268
00:44:26.790 --> 00:44:43.110
So we were restricting how far back. We can go in terms of what we're going to compare to. So we won't see these insane 225% wage increases or something like that. So we have done that in this in this dashboard and. Next slide please.

269
00:44:44.640 --> 00:44:51.240
So attain the living wage. This is affected by the calculation of the animal wages, obviously, because we take

270
00:44:52.620 --> 00:45:03.540
The annual wages and compared to those living wage in the district county of your college and that was basically given to us by the chancellor's office in terms of where

271
00:45:04.470 --> 00:45:19.890
Which County, we were to use for that calculation, and I think this is this informations already been communicated. You've already seen it in a live one but so the big, the big thing that happened was there was a huge increase in certain areas of our state in terms of living wage.

272
00:45:21.120 --> 00:45:35.640
When we did this metric in the first release. So just know that these are now quite a bit lower for certain areas like the Bay region and la and la and orange where the cost of living has just skyrocketed over the last two or three years mostly because of housing costs.

273
00:45:36.660 --> 00:45:38.970
And there's a few parts of the states were actually went down.

274
00:45:39.990 --> 00:45:45.120
But so you will see big hits in certain colleges in the Bay region and La orange a little bit in San Diego.

275
00:45:46.200 --> 00:45:57.480
But we are we are using what we think is the best calculation of the living wages, which is from the insights center there in the source that does a very good job of Kathleen a living wage for a single adult

276
00:45:58.890 --> 00:46:00.150
And. Next slide please.

277
00:46:06.060 --> 00:46:24.150
Okay, so that's where where I come in the next steps. As I mentioned, the release will be this week, and that the T is absolutely working as hard as they can right now to incorporate as many of these last minute.

278
00:46:25.230 --> 00:46:30.840
shoes that have been identified and improvements that we can make before we we release and

279
00:46:31.260 --> 00:46:45.660
If it was, I think, if it was up to the development team. They will, they will delay this until we got all the issues corrected. I just don't think that's possible. So we are going to get it out there on Friday and look forward to more of your feedback. Certainly.

280
00:46:47.070 --> 00:46:52.980
For the main release. There'll be our next. And I don't want to say final but certainly the major release.

281
00:46:53.910 --> 00:47:02.100
Where as I mentioned before, we have district micro regions and macro regions included will obviously add an additional year

282
00:47:02.850 --> 00:47:14.010
For transfer apprenticeship and employment data and then the cohort v for all the metrics and desegregation is will be included as well and I

283
00:47:14.970 --> 00:47:23.520
Carry that one that I read some of these questions I wanted to pose them to you because I think you know the answers to these that. So, if you don't mind. Yeah.

284
00:47:23.730 --> 00:47:35.190
It was a question about whether we using the 18 units certificates are we using any certificate or we have we integrated the 16 unit change, yet the response.

285
00:47:37.410 --> 00:47:49.440
Rate. So since we don't have any data mart the right, we don't have that 16 unit as a definition yet. So our definitions right are based on the data sets that were given

286
00:47:50.400 --> 00:48:03.030
So it. Yeah. So it's very clearly. Yeah, at a rate that I think we use 12 to 18 is one category, but we don't have that 16 or more unit category. Yeah, reported to us.

287
00:48:04.530 --> 00:48:12.690
So because we're basically getting you know what you're able to report to me. So when that 16 unit categories created and you're able to report that that we would get that data.

288
00:48:14.610 --> 00:48:14.970
Right.

289
00:48:16.890 --> 00:48:30.510
There was another question about a as category. And then there's another category called ATT and there's another category called all associate degrees. Can you respond to what the differences are there.

290
00:48:31.020 --> 00:48:39.090
Yes. So to be real clear so. So when we say a. So that means not for transfer and the way that we since there is not an

291
00:48:39.990 --> 00:48:52.110
Unfortunately, there is not a data element sp. Oh, who, as you most of you may know, in the call where researchers that would let us know that when you report, it's, it's a transfer and ATT

292
00:48:53.070 --> 00:48:58.200
So the way we determine if it's an ATT is by looking at a file, we've gotten from Kochi

293
00:48:59.010 --> 00:49:15.930
Actually from MIT is the ones that they use for the students that had funding for to determine which degrees are at and which are not. So you say AAS degree. That means they were not flagged as an ATT or transfer degree in Kochi system.

294
00:49:17.160 --> 00:49:21.090
And then, and that's where there might be some discrepancies with your internal data if that's

295
00:49:21.120 --> 00:49:33.690
out of sync. And of course, he, he was flagged as at your transfer by Kochi and then when we say associate or that would be the whole bucket. Right. Any, any associate degree at

296
00:49:34.110 --> 00:49:38.040
Yes. And of course, there's some, you know, of course duplication that might happen.

297
00:49:38.490 --> 00:49:52.770
To be really clear. So, I guess. Some students might earn as an agency. So that would there would be some duplication. So would not be exactly add up, you know, the as a as an 80s to get total associate degrees earner

298
00:49:57.720 --> 00:50:13.440
gravy. We know we'd really like to Ella, we know that the film would really like an element of them is that could flag, they could differentiate as from a team. We know that we need that flag in there though that's been that's been communicated to us, many times.

299
00:50:16.200 --> 00:50:27.780
Fair enough. Thank you, Karen. Just let me do a scan. Again, if he could scan with me as well. See if there any quick questions we can cover in the last couple of minutes that we have with our participants.

300
00:50:29.940 --> 00:50:38.640
Yeah, so there's less questions about the data mart headcount I think I copied it in the answer from a document that Kathy's prepared. That's pretty thorough

301
00:50:39.060 --> 00:50:46.140
Of why we still aren't going to match to data mart and we don't. We discussed this quite extensively with the chancellor's office and researchers and

302
00:50:47.280 --> 00:51:00.930
So somebody has put in there. I see what it does include which is really curious. The ABC or apps. So, it is the definition is excluding some students that take low unit certificate so there'd be a lot of like public safety would exclude

303
00:51:02.010 --> 00:51:13.050
There's a lot of non credit, you can see big differences between and well we'll have in the dashboard for non credit. We have looked extensively into those differences and

304
00:51:13.650 --> 00:51:29.730
We know that basically the big differences are due to the fact that that STP oh seven and data mart does not include all the categories that can afford it. If you want any more information on that, please, please contact me or please. I like the idea of going through that one.

305
00:51:31.050 --> 00:51:38.580
Research email because then I can, that's great because then we can put together one document. This is what we did before that was

306
00:51:39.120 --> 00:51:45.960
A lot of questions will get a lot of questions that ask the same question from different places. And then it's nice to respond to one place. But again,

307
00:51:46.920 --> 00:51:55.530
We are so we look forward if you wanted. If anything troubleshooting want to do with your own data, please. We look forward to working with you to help figure that out. We've done that with a few colleges

308
00:51:56.940 --> 00:51:59.250
And it's worked really well and it really helps us learn and

309
00:52:00.330 --> 00:52:02.100
Train. So it helps us figure out

310
00:52:04.320 --> 00:52:08.130
What to say, seen here from the questions.

311
00:52:09.300 --> 00:52:18.240
The one thing I wanted to add goes back again to the equity. So a number of other questions on that again. So I may be repeating myself, I apologize, but

312
00:52:19.140 --> 00:52:30.780
I know there, there, there were multiple ways to calculate the the equity numbers, whether you were using percentage point gaps or other methodologies.

313
00:52:31.350 --> 00:52:38.550
I don't believe there is consistency in the field as to how that should be calculated. I know that the chancellor's office released

314
00:52:38.820 --> 00:52:49.830
Last year, I believe the model that that they were going to put out there. We are going to the file that I shared with you that we're going to do all the pre calculations.

315
00:52:50.370 --> 00:53:05.220
So that you don't have to go to to the student success metrics and add up the columns will will try to have that ready for you will try to calculate it both ways that are the most common lead discussed

316
00:53:06.360 --> 00:53:19.200
In the field will have it both ways, calculated so that you can see that I don't think we have data on on some. We don't. We just don't have the data to do the percentage point gap calculations, but

317
00:53:20.130 --> 00:53:27.990
We'll do as many of the calculations as we can for you in that CSV, so that to to avoid the extra work that you have to do.

318
00:53:37.170 --> 00:53:51.900
We have just a couple more minutes. If you want to post any Karen, are we in the questions as fast as we can. But if there are any critical ones that we haven't responded to already, go ahead and post them again, we'll try to get a couple more of these is as we close up.

319
00:54:00.360 --> 00:54:02.610
And Karen, would you

320
00:54:03.720 --> 00:54:12.540
Introduce yourself again, more, more, more in more detail. I think people were asking. I think most people know know my name. But if you could go ahead and introduce yourself.

321
00:54:18.480 --> 00:54:20.220
Oh, sorry. I think we lost, Karen.

322
00:54:21.480 --> 00:54:31.530
Might have been a technical issue. Karen beltre Emma is a researcher who's been heavily involved with the dashboard work and she is with West and she's part of the

323
00:54:32.730 --> 00:54:43.950
implementation team. She's been with us since the beginning of this simplified metrics initiative work so quite knowledgeable as you could pick up hopefully from all the responses that she had

324
00:54:46.770 --> 00:54:52.410
All right. I think that's all the questions that I can. Oh, Karen. This back

325
00:54:53.430 --> 00:54:56.910
And can look like we've missed last year.

326
00:54:59.580 --> 00:55:06.540
Yeah, sorry, from what I do. So yeah, look at the booth. And yes, we work. We have a small team.

327
00:55:07.080 --> 00:55:14.700
Who's completely focused on and trying to help you understand what we've done on the dashboard and calculations. So we do communicate with our data scientists and quarters.

328
00:55:15.060 --> 00:55:26.850
A daily basis to make sure that we understand what they're doing and what's being displayed as the intent of the metric. So that is is really our job is to make sure we understand the intent of the metric and construct it in a way that's clearly communicated to the wall.

329
00:55:31.230 --> 00:55:43.440
Thank you, Karen, I think those are the main questions that I could parse through, we will scan through this again after the presentation. And again, we'll, we'll try to

330
00:55:43.770 --> 00:55:51.150
Do after issues you've identified will try to address them. If there are questions, we'll try to respond to them in the FAQ you

331
00:55:52.080 --> 00:56:02.910
Will try to include them in our discussions for the February 4 will try to integrate them in to our connecting the dots workshops that are coming up regionally.

332
00:56:03.420 --> 00:56:12.720
And it just in terms of closing up the webinar. And as I said, every last Wednesday of the month will do these

333
00:56:13.470 --> 00:56:30.600
Webinars if there are other webinars that you want to hear about the these webinars are for you truly so if if you want to, to make suggestions to me on other kinds of webinars we should have I'll be happy to do so.

334
00:56:32.130 --> 00:56:41.280
I tentatively said these based on the number of questions I get from the field. So certainly degree planning tools has been one and then

335
00:56:42.000 --> 00:56:58.260
I tentatively set this common era P because there's been a proof of concept done by and number of districts and colleges. So I was going to pose that as possible topic coming up. But if there are other ones you let me know if they're urgent wants to let me know that as well.

336
00:57:01.170 --> 00:57:08.700
Subscribe to the digital features newsletter. I know you can't see the length, but when you, when we, when we post this presentation, you'll be able to do that.

337
00:57:09.570 --> 00:57:24.270
And then you know who I am. There's my email address if you need to send me anything critical if it's down in the trenches data related items again send it to research at CC CC o.edu

338
00:57:25.200 --> 00:57:30.720
Thank you everyone for participating today. I hope these tech talks are useful for you.

339
00:57:31.380 --> 00:57:44.010
If you have any suggestions or improvements, send them my way. Thank you. Have a wonderful day can't thank you very much for your help today and for the CCC conferred team for providing the technical support in the background.

340
00:57:48.000 --> 00:57:50.370
Who by everyone will be ending this session now.